You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Minji Kim (Jira)" <ji...@apache.org> on 2023/04/12 08:28:00 UTC

[jira] [Created] (HBASE-27790) Data is deleted after the compaction and MOB cleaning operation

Minji Kim created HBASE-27790:
---------------------------------

             Summary: Data is deleted after the compaction and MOB cleaning operation
                 Key: HBASE-27790
                 URL: https://issues.apache.org/jira/browse/HBASE-27790
             Project: HBase
          Issue Type: Bug
    Affects Versions: 2.5.3
         Environment: hbase-2.5.3 (revision=d385524561f771dcb402905c2bdcaeb4a8fecbdb)

hadoop 2.10.2 (revision=965fd380006fa78b2315668fbc7eb432e1d8200f)
            Reporter: Minji Kim
         Attachments: original.png

I'm trying to save image binaries in the MOB column.
 
The column option is as follows:
{INDEX_BLOCK_ENCODING => 'NONE', MOB_THRESHOLD => '102400', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)}
 
I set the `hbase.master.mob.cleaner.period` as 3600 (1h) and called `major_compact` in the shell, 
to test if the compaction works as I expected.
As a result, StorefileSize became 0 MB without any other operations being done on the table.
Is it possible that the compaction job deletes all the data from the column?
 
The lost data size was about 150 TB and I am using 2000 region servers. (Data from other columns is fine)
File count from the MOB directory decreased from 525K to 78K after the compaction + mob cleaning.
 
I could not find anything about deleting data from the hbase logs, and it is unlikely that any other process deleted the data. (I wrote the data to the newly build hbase cluster)
 
I wonder if there can be a reason why all the data is gone.
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)