You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by David Larsen <dl...@aislea.com> on 2014/03/26 01:44:53 UTC
Deleted cells surviving compaction
My understanding of delete operations is that a delete essentially writes out a tombstone for the cell. Cells aren't actually deleted until a major compaction.
I performed the following sequence on a single cell:
1) Put
2) Delete with a timestamp of HConstants.LATEST_TIMESTAMP
3) Major compact table
4) Put
5) Do a regular scan or get... no value returned
6) Do a raw scan, and I see this:
myRowKey column=myColumn, timestamp=9223372036854775807, type=DeleteColumn
On one hand, I understand at a simple level why my HConstants.LATEST_TIMESTAMP on the Delete would create a sort of black hole where no future Put with a "real" timestamp would get in front of that timestamp. However, I'm curious why the tombstone appears not to get cleared out with the major compact. Am I just misunderstanding expected behavior?
--
version: 0.94.6-cdh4.5.0.
hbase(main):019:0> describe 'myTable'
DESCRIPTION ENABLED
{NAME => 'STAG_aiSnap2', FAMILIES => [{NAME => 'C', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE' true
, REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'SNAPPY', MIN_VERSIONS => '0', TTL => '21474
83647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'tr
ue', BLOCKCACHE => 'true'}]}
--
David Larsen
Software Engineer | Aisle A
dlarsen@AisleA.com<ma...@AisleA.com>
Re: Deleted cells surviving compaction
Posted by Ted Yu <yu...@gmail.com>.
Please take a look at HBASE-10118 'Major compact keeps deletes with future
timestamps'
Cheers
On Tue, Mar 25, 2014 at 5:44 PM, David Larsen <dl...@aislea.com> wrote:
> My understanding of delete operations is that a delete essentially writes
> out a tombstone for the cell. Cells aren't actually deleted until a major
> compaction.
>
> I performed the following sequence on a single cell:
>
> 1) Put
> 2) Delete with a timestamp of HConstants.LATEST_TIMESTAMP
> 3) Major compact table
> 4) Put
> 5) Do a regular scan or get... no value returned
> 6) Do a raw scan, and I see this:
>
> myRowKey column=myColumn,
> timestamp=9223372036854775807, type=DeleteColumn
>
> On one hand, I understand at a simple level why my
> HConstants.LATEST_TIMESTAMP on the Delete would create a sort of black hole
> where no future Put with a "real" timestamp would get in front of that
> timestamp. However, I'm curious why the tombstone appears not to get
> cleared out with the major compact. Am I just misunderstanding expected
> behavior?
>
> --
> version: 0.94.6-cdh4.5.0.
> hbase(main):019:0> describe 'myTable'
> DESCRIPTION
> ENABLED
> {NAME => 'STAG_aiSnap2', FAMILIES => [{NAME => 'C', DATA_BLOCK_ENCODING
> => 'NONE', BLOOMFILTER => 'NONE' true
> , REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'SNAPPY',
> MIN_VERSIONS => '0', TTL => '21474
> 83647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY =>
> 'false', ENCODE_ON_DISK => 'tr
> ue', BLOCKCACHE => 'true'}]}
>
> --
> David Larsen
> Software Engineer | Aisle A
> dlarsen@AisleA.com<ma...@AisleA.com>
>