You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "DieterDP-ng (via GitHub)" <gi...@apache.org> on 2025/10/06 09:26:10 UTC

[PR] HBASE-29604 BackupHFileCleaner uses flawed time based check [hbase]

DieterDP-ng opened a new pull request, #7360:
URL: https://github.com/apache/hbase/pull/7360

   Adds javadoc mentioning the concurrent usage and thread-safety need of FileCleanerDelegate#getDeletableFiles.
   
   Fixes a potential thread-safety issue in BackupHFileCleaner: this class tracks timestamps to block the deletion of recently loaded HFiles that might be needed for backup purposes. The timestamps were being registered from inside the concurrent method, which could result in recently added files getting deleted. Moved the timestamp registration to the postClean method, which is called only a single time per cleaner run, so recently loaded HFiles are in fact protected from deletion.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] HBASE-29604 BackupHFileCleaner uses flawed time based check [hbase]

Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7360:
URL: https://github.com/apache/hbase/pull/7360#issuecomment-3376911428

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |:----:|----------:|--------:|:--------:|:-------:|
   | +0 :ok: |  reexec  |   0m 37s |  |  Docker mode activated.  |
   |||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain any @author tags.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  |  Patch does not have any anti-patterns.  |
   |||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 19s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   5m 19s |  |  master passed  |
   | +1 :green_heart: |  compile  |   6m 24s |  |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 35s |  |  master passed  |
   | +1 :green_heart: |  spotbugs  |   3m 23s |  |  master passed  |
   | +1 :green_heart: |  spotless  |   1m 17s |  |  branch has no errors when running spotless:check.  |
   |||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 14s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   4m 52s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   5m 50s |  |  the patch passed  |
   | +1 :green_heart: |  javac  |   5m 50s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks issues.  |
   | +1 :green_heart: |  checkstyle  |   1m 20s |  |  the patch passed  |
   | +1 :green_heart: |  spotbugs  |   4m  8s |  |  the patch passed  |
   | +1 :green_heart: |  hadoopcheck  |  17m 35s |  |  Patch does not cause any errors with Hadoop 3.3.6 3.4.1.  |
   | +1 :green_heart: |  spotless  |   1m 13s |  |  patch has no errors when running spotless:check.  |
   |||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 28s |  |  The patch does not generate ASF License warnings.  |
   |  |   |  65m 21s |  |  |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/3/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/7360 |
   | Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
   | uname | Linux 63dca720c8f6 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 5d2e22d4da98db0a0c53ead86915bdb8792f8290 |
   | Default Java | Eclipse Adoptium-17.0.11+9 |
   | Max. process+thread count | 88 (vs. ulimit of 30000) |
   | modules | C: hbase-server hbase-backup U: . |
   | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/3/console |
   | versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
   | Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] HBASE-29604 BackupHFileCleaner uses flawed time based check [hbase]

Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7360:
URL: https://github.com/apache/hbase/pull/7360#issuecomment-3372319232

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |:----:|----------:|--------:|:--------:|:-------:|
   | +0 :ok: |  reexec  |   0m 41s |  |  Docker mode activated.  |
   |||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain any @author tags.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  |  Patch does not have any anti-patterns.  |
   |||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 18s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   4m 42s |  |  master passed  |
   | +1 :green_heart: |  compile  |   5m 20s |  |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m  0s |  |  master passed  |
   | +1 :green_heart: |  spotbugs  |   2m 24s |  |  master passed  |
   | +1 :green_heart: |  spotless  |   1m  1s |  |  branch has no errors when running spotless:check.  |
   |||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 12s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   4m  8s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   5m  4s |  |  the patch passed  |
   | +1 :green_heart: |  javac  |   5m  4s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks issues.  |
   | +1 :green_heart: |  checkstyle  |   1m 12s |  |  the patch passed  |
   | +1 :green_heart: |  spotbugs  |   2m 51s |  |  the patch passed  |
   | -1 :x: |  hadoopcheck  |   9m 49s |  |  The patch causes 15 errors with Hadoop v3.4.1.  |
   | +1 :green_heart: |  spotless  |   1m 21s |  |  patch has no errors when running spotless:check.  |
   |||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 54s |  |  The patch does not generate ASF License warnings.  |
   |  |   |  42m 59s |  |  |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/2/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/7360 |
   | Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
   | uname | Linux 9e42fb8262d0 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 5d2e22d4da98db0a0c53ead86915bdb8792f8290 |
   | Default Java | Eclipse Adoptium-17.0.11+9 |
   | hadoopcheck | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/2/artifact/yetus-general-check/output/patch-javac-3.4.1.txt |
   | Max. process+thread count | 88 (vs. ulimit of 30000) |
   | modules | C: hbase-server hbase-backup U: . |
   | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/2/console |
   | versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
   | Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] HBASE-29604 BackupHFileCleaner uses flawed time based check [hbase]

Posted by "ndimiduk (via GitHub)" <gi...@apache.org>.
ndimiduk commented on PR #7360:
URL: https://github.com/apache/hbase/pull/7360#issuecomment-3402409540

   It was a simple difference in collection utilities used on those lines, new in JDK11 APIs ... I've been away from HBase too long.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] HBASE-29604 BackupHFileCleaner uses flawed time based check [hbase]

Posted by "DieterDP-ng (via GitHub)" <gi...@apache.org>.
DieterDP-ng commented on code in PR #7360:
URL: https://github.com/apache/hbase/pull/7360#discussion_r2410615795


##########
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupHFileCleaner.java:
##########
@@ -52,10 +52,13 @@ public class BackupHFileCleaner extends BaseHFileCleanerDelegate implements Abor
   private boolean stopped = false;
   private boolean aborted = false;
   private Connection connection;
-  // timestamp of most recent read from backup system table
-  private long prevReadFromBackupTbl = 0;
-  // timestamp of 2nd most recent read from backup system table
-  private long secondPrevReadFromBackupTbl = 0;
+  // timestamp of most recent completed cleaning run
+  private volatile long previousCleaningCompletionTimestamp = 0;

Review Comment:
   Correct. That was also the (intended, but not actual) behavior of this cleaner before this PR I think.
   
   I guess we can just initialize this to X time before the current time. Could pick X rudimentary, I think a minute is sufficient, or we could set X to the interval in which the cleaner runs, if I trace the code correctly, that would be the value of `hbase.master.cleaner.interval`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] HBASE-29604 BackupHFileCleaner uses flawed time based check [hbase]

Posted by "DieterDP-ng (via GitHub)" <gi...@apache.org>.
DieterDP-ng commented on PR #7360:
URL: https://github.com/apache/hbase/pull/7360#issuecomment-3375771143

   Testing failures seem unrelated to the PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] HBASE-29604 BackupHFileCleaner uses flawed time based check [hbase]

Posted by "ndimiduk (via GitHub)" <gi...@apache.org>.
ndimiduk commented on code in PR #7360:
URL: https://github.com/apache/hbase/pull/7360#discussion_r2410446066


##########
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupHFileCleaner.java:
##########
@@ -52,10 +52,13 @@ public class BackupHFileCleaner extends BaseHFileCleanerDelegate implements Abor
   private boolean stopped = false;
   private boolean aborted = false;
   private Connection connection;
-  // timestamp of most recent read from backup system table
-  private long prevReadFromBackupTbl = 0;
-  // timestamp of 2nd most recent read from backup system table
-  private long secondPrevReadFromBackupTbl = 0;
+  // timestamp of most recent completed cleaning run
+  private volatile long previousCleaningCompletionTimestamp = 0;

Review Comment:
   So the first run of the chore during the process lifecycle will always do nothing? Is there something else we can base it's initialization value on?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] HBASE-29604 BackupHFileCleaner uses flawed time based check [hbase]

Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7360:
URL: https://github.com/apache/hbase/pull/7360#issuecomment-3370890572

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |:----:|----------:|--------:|:--------:|:-------:|
   | +0 :ok: |  reexec  |   0m 33s |  |  Docker mode activated.  |
   |||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain any @author tags.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  |  Patch does not have any anti-patterns.  |
   |||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 18s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   4m 17s |  |  master passed  |
   | +1 :green_heart: |  compile  |   4m 18s |  |  master passed  |
   | +1 :green_heart: |  checkstyle  |   0m 47s |  |  master passed  |
   | +1 :green_heart: |  spotbugs  |   2m  5s |  |  master passed  |
   | +1 :green_heart: |  spotless  |   0m 50s |  |  branch has no errors when running spotless:check.  |
   |||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 10s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m  1s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   3m 44s |  |  the patch passed  |
   | +1 :green_heart: |  javac  |   3m 44s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 45s |  |  the patch passed  |
   | +1 :green_heart: |  spotbugs  |   2m 12s |  |  the patch passed  |
   | +1 :green_heart: |  hadoopcheck  |  12m  3s |  |  Patch does not cause any errors with Hadoop 3.3.6 3.4.1.  |
   | +1 :green_heart: |  spotless  |   0m 49s |  |  patch has no errors when running spotless:check.  |
   |||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 21s |  |  The patch does not generate ASF License warnings.  |
   |  |   |  45m 16s |  |  |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/1/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/7360 |
   | Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
   | uname | Linux 01251cf326dc 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 93e93bc6148f27606c0348c621ba046dad0146d6 |
   | Default Java | Eclipse Adoptium-17.0.11+9 |
   | Max. process+thread count | 85 (vs. ulimit of 30000) |
   | modules | C: hbase-server hbase-backup U: . |
   | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/1/console |
   | versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
   | Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] HBASE-29604 BackupHFileCleaner uses flawed time based check [hbase]

Posted by "DieterDP-ng (via GitHub)" <gi...@apache.org>.
DieterDP-ng commented on PR #7360:
URL: https://github.com/apache/hbase/pull/7360#issuecomment-3370691148

   @ndimiduk @hgromer @rmdmattingly 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] HBASE-29604 BackupHFileCleaner uses flawed time based check [hbase]

Posted by "krconv (via GitHub)" <gi...@apache.org>.
krconv commented on code in PR #7360:
URL: https://github.com/apache/hbase/pull/7360#discussion_r2410934260


##########
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupHFileCleaner.java:
##########
@@ -52,10 +52,13 @@ public class BackupHFileCleaner extends BaseHFileCleanerDelegate implements Abor
   private boolean stopped = false;
   private boolean aborted = false;
   private Connection connection;
-  // timestamp of most recent read from backup system table
-  private long prevReadFromBackupTbl = 0;
-  // timestamp of 2nd most recent read from backup system table
-  private long secondPrevReadFromBackupTbl = 0;
+  // timestamp of most recent completed cleaning run
+  private volatile long previousCleaningCompletionTimestamp = 0;
+
+  @Override
+  public void postClean() {

Review Comment:
   I agree that this definitely looks like an oversight from the original implementation, and that this change corrects it to work as it seems it was originally intended



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] HBASE-29604 BackupHFileCleaner uses flawed time based check [hbase]

Posted by "ndimiduk (via GitHub)" <gi...@apache.org>.
ndimiduk merged PR #7360:
URL: https://github.com/apache/hbase/pull/7360


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] HBASE-29604 BackupHFileCleaner uses flawed time based check [hbase]

Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7360:
URL: https://github.com/apache/hbase/pull/7360#issuecomment-3377863744

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |:----:|----------:|--------:|:--------:|:-------:|
   | +0 :ok: |  reexec  |   0m 49s |  |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck  |
   |||| _ Prechecks _ |
   |||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 21s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   5m 43s |  |  master passed  |
   | +1 :green_heart: |  compile  |   2m 10s |  |  master passed  |
   | +1 :green_heart: |  javadoc  |   1m 21s |  |  master passed  |
   | +1 :green_heart: |  shadedjars  |  10m 17s |  |  branch has no errors when building our shaded downstream artifacts.  |
   |||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 28s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m 18s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   2m 15s |  |  the patch passed  |
   | +1 :green_heart: |  javac  |   2m 15s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 23s |  |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   9m 13s |  |  patch has no errors when building our shaded downstream artifacts.  |
   |||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 248m 59s |  |  hbase-server in the patch passed.  |
   | +1 :green_heart: |  unit  |  11m 19s |  |  hbase-backup in the patch passed.  |
   |  |   | 304m 38s |  |  |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/7360 |
   | Optional Tests | javac javadoc unit compile shadedjars |
   | uname | Linux dbdc8a3840cd 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 5d2e22d4da98db0a0c53ead86915bdb8792f8290 |
   | Default Java | Eclipse Adoptium-17.0.11+9 |
   |  Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/3/testReport/ |
   | Max. process+thread count | 4535 (vs. ulimit of 30000) |
   | modules | C: hbase-server hbase-backup U: . |
   | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/3/console |
   | versions | git=2.34.1 maven=3.9.8 |
   | Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] HBASE-29604 BackupHFileCleaner uses flawed time based check [hbase]

Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7360:
URL: https://github.com/apache/hbase/pull/7360#issuecomment-3373700370

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |:----:|----------:|--------:|:--------:|:-------:|
   | +0 :ok: |  reexec  |   0m 38s |  |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck  |
   |||| _ Prechecks _ |
   |||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 14s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   3m 57s |  |  master passed  |
   | +1 :green_heart: |  compile  |   1m 36s |  |  master passed  |
   | +1 :green_heart: |  javadoc  |   0m 50s |  |  master passed  |
   | +1 :green_heart: |  shadedjars  |   7m  2s |  |  branch has no errors when building our shaded downstream artifacts.  |
   |||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 13s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 48s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 23s |  |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 23s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 51s |  |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   7m 17s |  |  patch has no errors when building our shaded downstream artifacts.  |
   |||| _ Other Tests _ |
   | -1 :x: |  unit  | 255m 50s | [/patch-unit-hbase-server.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/2/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-server.txt) |  hbase-server in the patch failed.  |
   | +1 :green_heart: |  unit  |  12m  6s |  |  hbase-backup in the patch passed.  |
   |  |   | 300m 57s |  |  |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/7360 |
   | Optional Tests | javac javadoc unit compile shadedjars |
   | uname | Linux 3745f9007386 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 5d2e22d4da98db0a0c53ead86915bdb8792f8290 |
   | Default Java | Eclipse Adoptium-17.0.11+9 |
   |  Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/2/testReport/ |
   | Max. process+thread count | 3603 (vs. ulimit of 30000) |
   | modules | C: hbase-server hbase-backup U: . |
   | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/2/console |
   | versions | git=2.34.1 maven=3.9.8 |
   | Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] HBASE-29604 BackupHFileCleaner uses flawed time based check [hbase]

Posted by "ndimiduk (via GitHub)" <gi...@apache.org>.
ndimiduk commented on PR #7360:
URL: https://github.com/apache/hbase/pull/7360#issuecomment-3389180056

   Merged to master and backport to branch-3 is clean. Backport to branch-2 has minor conflict in the unit test. I think this means we've missed a backport along the way. It's a minor thing re: the `callCleaner()` helper method. @DieterDP-ng shall I naively resolve the conflict, or is there a commit we're missing -- do you mind taking a look?
   
   ```
   @@@ -108,12 -108,12 +108,21 @@@ public class TestBackupHFileCleaner 
         Iterable<FileStatus> deletable;
     
         // The first call will not allow any deletions because of the timestamp mechanism.
   ++<<<<<<< HEAD
    +    deletable = cleaner.getDeletableFiles(Arrays.asList(file1, file1Archived, file2, file3));
    +    assertEquals(Collections.emptySet(), Sets.newHashSet(deletable));
    +
    +    // No bulk loads registered, so all files can be deleted.
    +    deletable = cleaner.getDeletableFiles(Arrays.asList(file1, file1Archived, file2, file3));
    +    assertEquals(Sets.newHashSet(file1, file1Archived, file2, file3), Sets.newHashSet(deletable));
   ++=======
   +     deletable = callCleaner(cleaner, List.of(file1, file1Archived, file2, file3));
   +     assertEquals(Set.of(), Sets.newHashSet(deletable));
   + 
   +     // No bulk loads registered, so all files can be deleted.
   +     deletable = callCleaner(cleaner, List.of(file1, file1Archived, file2, file3));
   +     assertEquals(Set.of(file1, file1Archived, file2, file3), Sets.newHashSet(deletable));
   ++>>>>>>> 64436d408d (HBASE-29604 BackupHFileCleaner uses flawed time based check (#7360))
     
         // Register some bulk loads.
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] HBASE-29604 BackupHFileCleaner uses flawed time based check [hbase]

Posted by "DieterDP-ng (via GitHub)" <gi...@apache.org>.
DieterDP-ng commented on PR #7360:
URL: https://github.com/apache/hbase/pull/7360#issuecomment-3402204950

   > Merged to master and backport to branch-3 is clean. Backport to branch-2 has minor conflict in the unit test. I think this means we've missed a backport along the way. It's a minor thing re: the `callCleaner()` helper method. @DieterDP-ng shall I naively resolve the conflict, or is there a commit we're missing -- do you mind taking a look?
   
   Sorry for the late response, but I guess it's already figured out? Didn't quite get what the conflict was though.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] HBASE-29604 BackupHFileCleaner uses flawed time based check [hbase]

Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7360:
URL: https://github.com/apache/hbase/pull/7360#issuecomment-3371938523

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |:----:|----------:|--------:|:--------:|:-------:|
   | +0 :ok: |  reexec  |   0m 36s |  |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck  |
   |||| _ Prechecks _ |
   |||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 17s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   4m 10s |  |  master passed  |
   | +1 :green_heart: |  compile  |   1m 39s |  |  master passed  |
   | +1 :green_heart: |  javadoc  |   0m 56s |  |  master passed  |
   | +1 :green_heart: |  shadedjars  |   7m 43s |  |  branch has no errors when building our shaded downstream artifacts.  |
   |||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 14s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 45s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 34s |  |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 34s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 47s |  |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 38s |  |  patch has no errors when building our shaded downstream artifacts.  |
   |||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 243m 35s |  |  hbase-server in the patch passed.  |
   | -1 :x: |  unit  |  12m 21s | [/patch-unit-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/1/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-backup.txt) |  hbase-backup in the patch failed.  |
   |  |   | 289m 55s |  |  |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/7360 |
   | Optional Tests | javac javadoc unit compile shadedjars |
   | uname | Linux dcd39361d0a2 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 93e93bc6148f27606c0348c621ba046dad0146d6 |
   | Default Java | Eclipse Adoptium-17.0.11+9 |
   |  Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/1/testReport/ |
   | Max. process+thread count | 4188 (vs. ulimit of 30000) |
   | modules | C: hbase-server hbase-backup U: . |
   | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7360/1/console |
   | versions | git=2.34.1 maven=3.9.8 |
   | Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org