[jira] [Created] (HBASE-28568) Incremental backup set does not correctly shrink

["Dieter De Paepe (Jira)" <ji...@apache.org>]

Dieter De Paepe created HBASE-28568:
---------------------------------------

             Summary: Incremental backup set does not correctly shrink
                 Key: HBASE-28568
                 URL: https://issues.apache.org/jira/browse/HBASE-28568
             Project: HBase
          Issue Type: Bug
          Components: backup&restore
    Affects Versions: 2.6.0, 3.0.0
            Reporter: Dieter De Paepe


The logic in BackupAdminImpl#finalizeDelete does not properly clean up tables from the incrementalBackupTableSet (= the set of backups to include in every incremental backup).

This can lead to backups failing.

 

Minimal example to reproduce from source:
 * Add following to `conf/hbase-site.xml` to enable backups:

{code:java}
<property>
    <name>hbase.backup.enable</name>
    <value>true</value>
  </property>
  <property>
    <name>hbase.master.logcleaner.plugins</name>
    <value>org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner,org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner,org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner,org.apache.hadoop.hbase.backup.master.BackupLogCleaner</value>
  </property>
  <property>
    <name>hbase.procedure.master.classes</name>
    <value>org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager</value>
  </property>
  <property>
    <name>hbase.procedure.regionserver.classes</name>
    <value>org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager</value>
  </property>
  <property>
  <name>hbase.coprocessor.region.classes</name>
  <value>org.apache.hadoop.hbase.backup.BackupObserver</value>
</property>
  <property>
    <name>hbase.fs.tmp.dir</name>
    <value>file:/tmp/hbase-tmp</value>
  </property> {code}
 * Start HBase: {{bin/start-hbase.sh}}
 * 
{code:java}
echo "create 'table1', 'cf'" | bin/hbase shell -n
echo "create 'table2', 'cf'" | bin/hbase shell -nbin/hbase backup create full file:/tmp/hbasebackups -t table1
bin/hbase backup create full file:/tmp/hbasebackups -t table2
bin/hbase backup create incremental file:/tmp/hbasebackups
# Deletes the 2 most recent backups
bin/hbase backup delete -l $(bin/hbase backup history | head -n1  | tail -n -1 | grep -o -P "backup_\d+"),$(bin/hbase backup history | head -n2  | tail -n -1 | grep -o -P "backup_\d+")
bin/hbase backup create incremental file:/tmp/hbasebackups -t table1

[...]
2024-05-06T14:28:46,420 INFO  [main {}] mapreduce.MapReduceBackupCopyJob: Progress: 100.0% subTask: 1.0 mapProgress: 1.0
2024-05-06T14:28:46,468 ERROR [main {}] backup.BackupDriver: Error running command-line tool
java.lang.IllegalStateException: Unable to find full backup that contains tables: [table2]
    at org.apache.hadoop.hbase.backup.impl.BackupManager.getAncestors(BackupManager.java:323) ~[hbase-backup-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
    at org.apache.hadoop.hbase.backup.impl.BackupManager.getAncestors(BackupManager.java:336) ~[hbase-backup-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
    at org.apache.hadoop.hbase.backup.impl.TableBackupClient.addManifest(TableBackupClient.java:286) ~[hbase-backup-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
    at org.apache.hadoop.hbase.backup.impl.TableBackupClient.completeBackup(TableBackupClient.java:351) ~[hbase-backup-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
    at org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.execute(IncrementalTableBackupClient.java:313) ~[hbase-backup-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
    at org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.backupTables(BackupAdminImpl.java:603) ~[hbase-backup-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
    at org.apache.hadoop.hbase.backup.impl.BackupCommands$CreateCommand.execute(BackupCommands.java:345) ~[hbase-backup-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
    at org.apache.hadoop.hbase.backup.BackupDriver.parseAndRun(BackupDriver.java:134) ~[hbase-backup-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
    at org.apache.hadoop.hbase.backup.BackupDriver.doWork(BackupDriver.java:169) ~[hbase-backup-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
    at org.apache.hadoop.hbase.backup.BackupDriver.run(BackupDriver.java:199) ~[hbase-backup-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:82) ~[hadoop-common-3.3.5.jar:?]
    at org.apache.hadoop.hbase.backup.BackupDriver.main(BackupDriver.java:177) ~[hbase-backup-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
{code}

PR will follow soon.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)