You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "vinayakphegde (via GitHub)" <gi...@apache.org> on 2023/11/17 13:31:21 UTC
[PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
vinayakphegde opened a new pull request, #5528:
URL: https://github.com/apache/hbase/pull/5528
This task introduces a new JMX metric to expose the size of the 'oldWALs' directory. To calculate the directory size, a ScheduledChore is implemented, allowing for efficient computation over a potentially time-consuming operation. The ScheduledChore updates the metrics at 5-minute intervals.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5528:
URL: https://github.com/apache/hbase/pull/5528#issuecomment-1835940180
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 34s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 41s | master passed |
| +1 :green_heart: | compile | 4m 50s | master passed |
| +1 :green_heart: | checkstyle | 1m 6s | master passed |
| +0 :ok: | refguide | 2m 48s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. |
| +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 9m 39s | master passed |
| -0 :warning: | patch | 7m 45s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 41s | the patch passed |
| +1 :green_heart: | compile | 4m 44s | the patch passed |
| +1 :green_heart: | javac | 4m 44s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 6s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +0 :ok: | refguide | 2m 13s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. |
| +1 :green_heart: | hadoopcheck | 9m 22s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. |
| +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 10m 8s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 41s | The patch does not generate ASF License warnings. |
| | | 61m 5s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/5/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5528 |
| JIRA Issue | HBASE-28209 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile refguide xml |
| uname | Linux 968724d4c239 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 7f3e400268 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| refguide | https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5528/5/yetus-general-check/output/branch-site/book.html |
| refguide | https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5528/5/yetus-general-check/output/patch-site/book.html |
| Max. process+thread count | 176 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-hadoop-compat hbase-server . U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/5/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5528:
URL: https://github.com/apache/hbase/pull/5528#issuecomment-1816954010
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 34s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 54s | master passed |
| +1 :green_heart: | compile | 2m 46s | master passed |
| +1 :green_heart: | checkstyle | 0m 45s | master passed |
| +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 1m 54s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 42s | the patch passed |
| +1 :green_heart: | compile | 2m 43s | the patch passed |
| +1 :green_heart: | javac | 2m 43s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 45s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 9m 23s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. |
| +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 2m 7s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 20s | The patch does not generate ASF License warnings. |
| | | 34m 48s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5528 |
| JIRA Issue | HBASE-28209 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux 81f73d9d935e 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / a6bc210762 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Max. process+thread count | 78 (vs. ulimit of 30000) |
| modules | C: hbase-hadoop-compat hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/2/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5528:
URL: https://github.com/apache/hbase/pull/5528#issuecomment-1836451385
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 32s | master passed |
| +1 :green_heart: | compile | 1m 34s | master passed |
| +1 :green_heart: | shadedjars | 5m 12s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 2m 3s | master passed |
| -0 :warning: | patch | 7m 39s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 19s | the patch passed |
| +1 :green_heart: | compile | 1m 36s | the patch passed |
| +1 :green_heart: | javac | 1m 36s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 15s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 2m 4s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 350m 47s | root in the patch failed. |
| | | 379m 26s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5528 |
| JIRA Issue | HBASE-28209 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux d2e71cbcd399 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 7f3e400268 |
| Default Java | Temurin-1.8.0_352-b08 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/5/testReport/ |
| Max. process+thread count | 5126 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-hadoop-compat hbase-server . U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/5/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5528:
URL: https://github.com/apache/hbase/pull/5528#issuecomment-1819515189
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 40s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 37s | master passed |
| +1 :green_heart: | compile | 1m 55s | master passed |
| +1 :green_heart: | shadedjars | 4m 51s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 2m 46s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 45s | the patch passed |
| +1 :green_heart: | compile | 1m 56s | the patch passed |
| +1 :green_heart: | javac | 1m 56s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 53s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 2m 45s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 294m 7s | root in the patch failed. |
| | | 327m 0s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5528 |
| JIRA Issue | HBASE-28209 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux eb582b702c68 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 9631af791e |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/3/testReport/ |
| Max. process+thread count | 8316 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-hadoop-compat hbase-server . U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/3/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "wchevreuil (via GitHub)" <gi...@apache.org>.
wchevreuil commented on code in PR #5528:
URL: https://github.com/apache/hbase/pull/5528#discussion_r1397382901
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterWalManager.java:
##########
@@ -398,4 +414,33 @@ public void archiveMetaLog(final ServerName serverName) {
LOG.warn("Failed archiving meta log for server " + serverName, ie);
}
}
+
+ private static Stoppable createDummyStoppable() {
+ return new Stoppable() {
+ private volatile boolean isStopped = false;
+
+ @Override
+ public void stop(String why) {
+ isStopped = true;
+ }
+
+ @Override
+ public boolean isStopped() {
+ return isStopped;
+ }
+ };
+ }
+
+ public ScheduledChore getOldWALsDirSizeUpdaterChore() {
Review Comment:
Can we have this Chore imlemented on its own class? That way we don't need to define new methods on MasterWalManager that just concerns this Chore logic, like the `createDummyStoppable` and `updateWalDirSize()` above.
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterWalManager.java:
##########
@@ -398,4 +414,33 @@ public void archiveMetaLog(final ServerName serverName) {
LOG.warn("Failed archiving meta log for server " + serverName, ie);
}
}
+
+ private static Stoppable createDummyStoppable() {
+ return new Stoppable() {
+ private volatile boolean isStopped = false;
+
+ @Override
+ public void stop(String why) {
+ isStopped = true;
+ }
+
+ @Override
+ public boolean isStopped() {
+ return isStopped;
+ }
+ };
+ }
+
+ public ScheduledChore getOldWALsDirSizeUpdaterChore() {
+ return new ScheduledChore("UpdateOldWALsDirSize", createDummyStoppable(), OLD_WAL_DIR_UPDATE_INTERVAL) {
Review Comment:
Can we make this interval configurable? This way we could also give the option to disable this chore altogether.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5528:
URL: https://github.com/apache/hbase/pull/5528#issuecomment-1816497389
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 12s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 49s | master passed |
| +1 :green_heart: | compile | 2m 46s | master passed |
| +1 :green_heart: | checkstyle | 0m 45s | master passed |
| +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 1m 51s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 41s | the patch passed |
| +1 :green_heart: | compile | 2m 45s | the patch passed |
| +1 :green_heart: | javac | 2m 45s | the patch passed |
| -0 :warning: | checkstyle | 0m 33s | hbase-server: The patch generated 2 new + 8 unchanged - 0 fixed = 10 total (was 8) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 9m 19s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. |
| -1 :x: | spotless | 0m 15s | patch has 24 errors when running spotless:check, run spotless:apply to fix. |
| +1 :green_heart: | spotbugs | 2m 5s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 20s | The patch does not generate ASF License warnings. |
| | | 33m 42s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5528 |
| JIRA Issue | HBASE-28209 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux daefbfc8466f 5.4.0-153-generic #170-Ubuntu SMP Fri Jun 16 13:43:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 9e74cc0d65 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/1/artifact/yetus-general-check/output/patch-spotless.txt |
| Max. process+thread count | 79 (vs. ulimit of 30000) |
| modules | C: hbase-hadoop-compat hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/1/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "wchevreuil (via GitHub)" <gi...@apache.org>.
wchevreuil commented on code in PR #5528:
URL: https://github.com/apache/hbase/pull/5528#discussion_r1398975901
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterWalManager.java:
##########
@@ -398,4 +414,33 @@ public void archiveMetaLog(final ServerName serverName) {
LOG.warn("Failed archiving meta log for server " + serverName, ie);
}
}
+
+ private static Stoppable createDummyStoppable() {
+ return new Stoppable() {
+ private volatile boolean isStopped = false;
+
+ @Override
+ public void stop(String why) {
+ isStopped = true;
+ }
+
+ @Override
+ public boolean isStopped() {
+ return isStopped;
+ }
+ };
+ }
+
+ public ScheduledChore getOldWALsDirSizeUpdaterChore() {
+ return new ScheduledChore("UpdateOldWALsDirSize", createDummyStoppable(), OLD_WAL_DIR_UPDATE_INTERVAL) {
Review Comment:
We could return 0 or a negative value in the metric, and document that such values mean the metric calculation is disabled. I think it's worth to make it possible for operators to disable it because this would consume a spot in the ChoreExecutor pool, which could impact execution of other important background work, such as the file cleaners.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5528:
URL: https://github.com/apache/hbase/pull/5528#issuecomment-1819031955
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 11s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 38s | master passed |
| +1 :green_heart: | compile | 4m 48s | master passed |
| +1 :green_heart: | checkstyle | 1m 6s | master passed |
| +0 :ok: | refguide | 2m 45s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. |
| +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 9m 38s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 37s | the patch passed |
| +1 :green_heart: | compile | 4m 51s | the patch passed |
| +1 :green_heart: | javac | 4m 51s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 5s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. |
| +0 :ok: | refguide | 2m 15s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. |
| +1 :green_heart: | hadoopcheck | 9m 15s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. |
| +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 10m 14s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 40s | The patch does not generate ASF License warnings. |
| | | 61m 1s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5528 |
| JIRA Issue | HBASE-28209 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile refguide xml |
| uname | Linux fd7b08272f5d 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 9631af791e |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| refguide | https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5528/3/yetus-general-check/output/branch-site/book.html |
| refguide | https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5528/3/yetus-general-check/output/patch-site/book.html |
| Max. process+thread count | 178 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-hadoop-compat hbase-server . U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/3/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "vinayakphegde (via GitHub)" <gi...@apache.org>.
vinayakphegde commented on code in PR #5528:
URL: https://github.com/apache/hbase/pull/5528#discussion_r1408799214
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterWalManager.java:
##########
@@ -89,6 +89,12 @@ public boolean accept(Path p) {
// create the split log lock
private final Lock splitLogLock = new ReentrantLock();
+ // old WALs directory size in bytes
+ private long oldWALsDirSize;
+
+ // old WALs directory size calculation interval
+ private final int OLD_WAL_DIR_UPDATE_INTERVAL = 5 * 60 * 1000; // 5 mins
Review Comment:
It is not used now. Will remove it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5528:
URL: https://github.com/apache/hbase/pull/5528#issuecomment-1831804525
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 34s | master passed |
| +1 :green_heart: | compile | 6m 33s | master passed |
| +1 :green_heart: | checkstyle | 1m 19s | master passed |
| +0 :ok: | refguide | 3m 18s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. |
| +1 :green_heart: | spotless | 0m 50s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 12m 28s | master passed |
| -0 :warning: | patch | 2m 5s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 28s | the patch passed |
| +1 :green_heart: | compile | 6m 45s | the patch passed |
| +1 :green_heart: | javac | 6m 45s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 18s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. |
| +0 :ok: | refguide | 2m 53s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. |
| +1 :green_heart: | hadoopcheck | 11m 19s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. |
| +1 :green_heart: | spotless | 0m 56s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 13m 46s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 40s | The patch does not generate ASF License warnings. |
| | | 77m 57s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5528 |
| JIRA Issue | HBASE-28209 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile refguide xml |
| uname | Linux a4e2d430adde 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 44eb408a6d |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| refguide | https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5528/4/yetus-general-check/output/branch-site/book.html |
| refguide | https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5528/4/yetus-general-check/output/patch-site/book.html |
| Max. process+thread count | 177 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-hadoop-compat hbase-server . U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/4/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "vinayakphegde (via GitHub)" <gi...@apache.org>.
vinayakphegde commented on code in PR #5528:
URL: https://github.com/apache/hbase/pull/5528#discussion_r1399010206
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterWalManager.java:
##########
@@ -398,4 +414,33 @@ public void archiveMetaLog(final ServerName serverName) {
LOG.warn("Failed archiving meta log for server " + serverName, ie);
}
}
+
+ private static Stoppable createDummyStoppable() {
+ return new Stoppable() {
+ private volatile boolean isStopped = false;
+
+ @Override
+ public void stop(String why) {
+ isStopped = true;
+ }
+
+ @Override
+ public boolean isStopped() {
+ return isStopped;
+ }
+ };
+ }
+
+ public ScheduledChore getOldWALsDirSizeUpdaterChore() {
+ return new ScheduledChore("UpdateOldWALsDirSize", createDummyStoppable(), OLD_WAL_DIR_UPDATE_INTERVAL) {
Review Comment:
Got it, will modify the code for that!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "vinayakphegde (via GitHub)" <gi...@apache.org>.
vinayakphegde commented on code in PR #5528:
URL: https://github.com/apache/hbase/pull/5528#discussion_r1398666429
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterWalManager.java:
##########
@@ -398,4 +414,33 @@ public void archiveMetaLog(final ServerName serverName) {
LOG.warn("Failed archiving meta log for server " + serverName, ie);
}
}
+
+ private static Stoppable createDummyStoppable() {
+ return new Stoppable() {
+ private volatile boolean isStopped = false;
+
+ @Override
+ public void stop(String why) {
+ isStopped = true;
+ }
+
+ @Override
+ public boolean isStopped() {
+ return isStopped;
+ }
+ };
+ }
+
+ public ScheduledChore getOldWALsDirSizeUpdaterChore() {
+ return new ScheduledChore("UpdateOldWALsDirSize", createDummyStoppable(), OLD_WAL_DIR_UPDATE_INTERVAL) {
Review Comment:
If we disable it, won't we encounter issues with the incorrect results of the oldWALs directory size JMX metrics? Could that potentially be a problem? That's why I didn't make it configurable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5528:
URL: https://github.com/apache/hbase/pull/5528#issuecomment-1832551541
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 42s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 48s | master passed |
| +1 :green_heart: | compile | 1m 59s | master passed |
| +1 :green_heart: | shadedjars | 6m 3s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 2m 45s | master passed |
| -0 :warning: | patch | 9m 15s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 9s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 0s | the patch passed |
| +1 :green_heart: | compile | 2m 2s | the patch passed |
| +1 :green_heart: | javac | 2m 2s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 45s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 2m 37s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 451m 5s | root in the patch failed. |
| | | 486m 10s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5528 |
| JIRA Issue | HBASE-28209 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 836c7439e3d9 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 44eb408a6d |
| Default Java | Temurin-1.8.0_352-b08 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/4/testReport/ |
| Max. process+thread count | 7388 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-hadoop-compat hbase-server . U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/4/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5528:
URL: https://github.com/apache/hbase/pull/5528#issuecomment-1832440672
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 43s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 8s | master passed |
| +1 :green_heart: | compile | 3m 6s | master passed |
| +1 :green_heart: | shadedjars | 7m 20s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 4m 26s | master passed |
| -0 :warning: | patch | 12m 18s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 16s | the patch passed |
| +1 :green_heart: | compile | 2m 54s | the patch passed |
| +1 :green_heart: | javac | 2m 54s | the patch passed |
| +1 :green_heart: | shadedjars | 7m 17s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 4m 13s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 364m 50s | root in the patch failed. |
| | | 410m 59s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5528 |
| JIRA Issue | HBASE-28209 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux c9f053dc3bdf 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 44eb408a6d |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/4/testReport/ |
| Max. process+thread count | 7944 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-hadoop-compat hbase-server . U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/4/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5528:
URL: https://github.com/apache/hbase/pull/5528#issuecomment-1816955043
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 43s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 54s | master passed |
| +1 :green_heart: | compile | 1m 3s | master passed |
| +1 :green_heart: | shadedjars | 4m 55s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 42s | the patch passed |
| +1 :green_heart: | compile | 1m 3s | the patch passed |
| +1 :green_heart: | javac | 1m 3s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 48s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 40s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 42s | hbase-hadoop-compat in the patch passed. |
| -1 :x: | unit | 13m 35s | hbase-server in the patch failed. |
| | | 36m 2s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5528 |
| JIRA Issue | HBASE-28209 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux c34e2f009c71 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / a6bc210762 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/2/testReport/ |
| Max. process+thread count | 2039 (vs. ulimit of 30000) |
| modules | C: hbase-hadoop-compat hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/2/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "wchevreuil (via GitHub)" <gi...@apache.org>.
wchevreuil merged PR #5528:
URL: https://github.com/apache/hbase/pull/5528
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5528:
URL: https://github.com/apache/hbase/pull/5528#issuecomment-1816910397
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 13s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 54s | master passed |
| +1 :green_heart: | compile | 1m 3s | master passed |
| +1 :green_heart: | shadedjars | 4m 53s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 41s | the patch passed |
| +1 :green_heart: | compile | 1m 3s | the patch passed |
| +1 :green_heart: | javac | 1m 3s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 46s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 46s | hbase-hadoop-compat in the patch passed. |
| -1 :x: | unit | 276m 24s | hbase-server in the patch failed. |
| | | 302m 31s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5528 |
| JIRA Issue | HBASE-28209 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux de89f0609d7d 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 9e74cc0d65 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/1/testReport/ |
| Max. process+thread count | 4827 (vs. ulimit of 30000) |
| modules | C: hbase-hadoop-compat hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/1/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5528:
URL: https://github.com/apache/hbase/pull/5528#issuecomment-1816904349
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 40s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 42s | master passed |
| +1 :green_heart: | compile | 1m 4s | master passed |
| +1 :green_heart: | shadedjars | 6m 5s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 44s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 0s | the patch passed |
| +1 :green_heart: | compile | 1m 4s | the patch passed |
| +1 :green_heart: | javac | 1m 4s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 28s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 38s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 45s | hbase-hadoop-compat in the patch passed. |
| -1 :x: | unit | 269m 27s | hbase-server in the patch failed. |
| | | 297m 56s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5528 |
| JIRA Issue | HBASE-28209 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 9d02e497a915 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 9e74cc0d65 |
| Default Java | Temurin-1.8.0_352-b08 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/1/testReport/ |
| Max. process+thread count | 5197 (vs. ulimit of 30000) |
| modules | C: hbase-hadoop-compat hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/1/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5528:
URL: https://github.com/apache/hbase/pull/5528#issuecomment-1819586282
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 7s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 31s | master passed |
| +1 :green_heart: | compile | 1m 34s | master passed |
| +1 :green_heart: | shadedjars | 5m 10s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 2m 3s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 17s | the patch passed |
| +1 :green_heart: | compile | 1m 30s | the patch passed |
| +1 :green_heart: | javac | 1m 30s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 8s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 2m 2s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 347m 24s | root in the patch failed. |
| | | 376m 56s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5528 |
| JIRA Issue | HBASE-28209 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 5f2e34090759 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 9631af791e |
| Default Java | Temurin-1.8.0_352-b08 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/3/testReport/ |
| Max. process+thread count | 5570 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-hadoop-compat hbase-server . U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/3/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5528:
URL: https://github.com/apache/hbase/pull/5528#issuecomment-1836378941
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 39s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 50s | master passed |
| +1 :green_heart: | compile | 2m 0s | master passed |
| +1 :green_heart: | shadedjars | 4m 53s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 2m 47s | master passed |
| -0 :warning: | patch | 8m 8s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 39s | the patch passed |
| +1 :green_heart: | compile | 1m 57s | the patch passed |
| +1 :green_heart: | javac | 1m 57s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 52s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 2m 49s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 310m 10s | root in the patch passed. |
| | | 343m 12s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5528 |
| JIRA Issue | HBASE-28209 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux d43c094e0126 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 7f3e400268 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/5/testReport/ |
| Max. process+thread count | 8322 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-hadoop-compat hbase-server . U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/5/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "wchevreuil (via GitHub)" <gi...@apache.org>.
wchevreuil commented on code in PR #5528:
URL: https://github.com/apache/hbase/pull/5528#discussion_r1407961823
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterWalManager.java:
##########
@@ -89,6 +89,12 @@ public boolean accept(Path p) {
// create the split log lock
private final Lock splitLogLock = new ReentrantLock();
+ // old WALs directory size in bytes
+ private long oldWALsDirSize;
+
+ // old WALs directory size calculation interval
+ private final int OLD_WAL_DIR_UPDATE_INTERVAL = 5 * 60 * 1000; // 5 mins
Review Comment:
Where is this used?
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/OldWALsDirSizeUpdaterChore.java:
##########
Review Comment:
Nit: let's call it "OldWALsDirSizeChore"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "vinayakphegde (via GitHub)" <gi...@apache.org>.
vinayakphegde commented on code in PR #5528:
URL: https://github.com/apache/hbase/pull/5528#discussion_r1398666808
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterWalManager.java:
##########
@@ -398,4 +414,33 @@ public void archiveMetaLog(final ServerName serverName) {
LOG.warn("Failed archiving meta log for server " + serverName, ie);
}
}
+
+ private static Stoppable createDummyStoppable() {
+ return new Stoppable() {
+ private volatile boolean isStopped = false;
+
+ @Override
+ public void stop(String why) {
+ isStopped = true;
+ }
+
+ @Override
+ public boolean isStopped() {
+ return isStopped;
+ }
+ };
+ }
+
+ public ScheduledChore getOldWALsDirSizeUpdaterChore() {
Review Comment:
Sure, Will do that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-28209: Create a jmx metrics to expose the oldWALs directory size [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5528:
URL: https://github.com/apache/hbase/pull/5528#issuecomment-1817216558
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 26s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 30s | master passed |
| +1 :green_heart: | compile | 0m 46s | master passed |
| +1 :green_heart: | shadedjars | 5m 12s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 32s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 19s | the patch passed |
| +1 :green_heart: | compile | 0m 47s | the patch passed |
| +1 :green_heart: | javac | 0m 47s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 7s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 30s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 33s | hbase-hadoop-compat in the patch passed. |
| -1 :x: | unit | 223m 59s | hbase-server in the patch failed. |
| | | 247m 14s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5528 |
| JIRA Issue | HBASE-28209 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 69545fb92888 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / a6bc210762 |
| Default Java | Temurin-1.8.0_352-b08 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/2/testReport/ |
| Max. process+thread count | 4843 (vs. ulimit of 30000) |
| modules | C: hbase-hadoop-compat hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5528/2/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org