[jira] [Created] (HBASE-29604) BackupHFileCleaner uses flawed time based check

["Dieter De Paepe (Jira)" <ji...@apache.org>]

Dieter De Paepe created HBASE-29604:
---------------------------------------

             Summary: BackupHFileCleaner uses flawed time based check
                 Key: HBASE-29604
                 URL: https://issues.apache.org/jira/browse/HBASE-29604
             Project: HBase
          Issue Type: Bug
          Components: backup&restore
    Affects Versions: 3.0.0-beta-1, 2.6.0, 4.0.0-alpha-1
            Reporter: Dieter De Paepe


BackupHFileCleaner is responsible for preventing the cleanup of bulkloaded HFiles that are still required by the backup & restore mechanism. It does this using 2 checks:
 * The backupsystemtable stores which HFile bulk loads are required for the next incremental backup. Any HFile present here cannot be deleted.
 * A time-based check is present to avoid recently created HFiles from being deleted. The intention is to avoid deletion of HFiles newer than the previous run of the cleaner. I believe is to avoid race conditions between the cleaner and entries in the backupsystemtable that get created while the cleaner is running.

In a single-threaded context, this works correctly.

However, the cleaner is actually used concurrently in the hfile_cleaner-dir-scan-pool to scan multiple subdirectories in `CleanerChore#traverseAndDelete` (line 492). This means the time-based check is not guaranteed to protect recently created HFiles. This has a (small) chance to cause data loss (in a backup) if an HFile is wrongfully deleted.

I also strongly suggest to add a mention to FileCleanerDelegate that implementations should be thread-safe.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)