You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "hchaverri (via GitHub)" <gi...@apache.org> on 2023/06/03 19:47:51 UTC
[GitHub] [hadoop] hchaverri opened a new pull request, #5714: HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency
hchaverri opened a new pull request, #5714:
URL: https://github.com/apache/hadoop/pull/5714
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g., 'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
Equivalent PR for master: https://github.com/apache/hadoop/pull/5691
Changes to the NamenodeHeartbeatService to update JMX reports on a configurable frequency or to disable JMX updates altogether, as opposed to updating it every time the service wakes up. This will help reduce the load on Namenodes in clusters with a high number of routers.
With these changes, we expect the NamenodeStatusReport to contain slightly outdated Namenode information, which should not impact critical paths for routers.
### How was this patch tested?
Added unit test on TestRouterNamenodeMonitoring#testJmxRequestFrequency that tests the following configurations:
Default configuration which should update JMX every time the report is generated.
Configuration with JMX updates disabled, which should never update JMX
Configuration with a high JMX update interval, which should not update JMX every time the report is generated.
### For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [X] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [X] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [X] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hchaverri closed pull request #5714: HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency
Posted by "hchaverri (via GitHub)" <gi...@apache.org>.
hchaverri closed pull request #5714: HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency
URL: https://github.com/apache/hadoop/pull/5714
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5714: HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5714:
URL: https://github.com/apache/hadoop/pull/5714#issuecomment-1654872092
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 10m 22s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ branch-3.3 Compile Tests _ |
| +1 :green_heart: | mvninstall | 51m 45s | | branch-3.3 passed |
| +1 :green_heart: | compile | 0m 42s | | branch-3.3 passed |
| +1 :green_heart: | checkstyle | 0m 28s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 0m 43s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 0m 59s | | branch-3.3 passed |
| +1 :green_heart: | spotbugs | 1m 23s | | branch-3.3 passed |
| +1 :green_heart: | shadedclient | 40m 34s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed |
| +1 :green_heart: | spotbugs | 1m 22s | | the patch passed |
| +1 :green_heart: | shadedclient | 40m 51s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 16m 48s | | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not generate ASF License warnings. |
| | | 172m 30s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5714/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5714 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 11638c3d655f 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3 / f66581e9c9ab60946db5fe327ed447feed34d1bb |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~18.04-b09 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5714/2/testReport/ |
| Max. process+thread count | 2199 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5714/2/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5714: HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5714:
URL: https://github.com/apache/hadoop/pull/5714#issuecomment-1575217379
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 10m 14s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ branch-3.3 Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 44s | | branch-3.3 passed |
| +1 :green_heart: | compile | 0m 34s | | branch-3.3 passed |
| +1 :green_heart: | checkstyle | 0m 28s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 0m 39s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 1m 4s | | branch-3.3 passed |
| +1 :green_heart: | spotbugs | 1m 29s | | branch-3.3 passed |
| +1 :green_heart: | shadedclient | 26m 22s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 16s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 48s | | the patch passed |
| +1 :green_heart: | spotbugs | 1m 18s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 46s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 16m 28s | | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not generate ASF License warnings. |
| | | 129m 25s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5714/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5714 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 3723a72f1d41 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3 / f66581e9c9ab60946db5fe327ed447feed34d1bb |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~18.04-b09 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5714/1/testReport/ |
| Max. process+thread count | 2208 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5714/1/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a diff in pull request #5714: HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency
Posted by "goiri (via GitHub)" <gi...@apache.org>.
goiri commented on code in PR #5714:
URL: https://github.com/apache/hadoop/pull/5714#discussion_r1218675945
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/NamenodeHeartbeatService.java:
##########
@@ -348,44 +362,73 @@ private void updateJMXParameters(
String address, NamenodeStatusReport report) {
try {
// TODO part of this should be moved to its own utility
- String query = "Hadoop:service=NameNode,name=FSNamesystem*";
- JSONArray aux = FederationUtil.getJmx(
- query, address, connectionFactory, scheme);
- if (aux != null) {
- for (int i = 0; i < aux.length(); i++) {
- JSONObject jsonObject = aux.getJSONObject(i);
- String name = jsonObject.getString("name");
- if (name.equals("Hadoop:service=NameNode,name=FSNamesystemState")) {
- report.setDatanodeInfo(
- jsonObject.getInt("NumLiveDataNodes"),
- jsonObject.getInt("NumDeadDataNodes"),
- jsonObject.getInt("NumStaleDataNodes"),
- jsonObject.getInt("NumDecommissioningDataNodes"),
- jsonObject.getInt("NumDecomLiveDataNodes"),
- jsonObject.getInt("NumDecomDeadDataNodes"),
- jsonObject.optInt("NumInMaintenanceLiveDataNodes"),
- jsonObject.optInt("NumInMaintenanceDeadDataNodes"),
- jsonObject.optInt("NumEnteringMaintenanceDataNodes"));
- } else if (name.equals(
- "Hadoop:service=NameNode,name=FSNamesystem")) {
- report.setNamesystemInfo(
- jsonObject.getLong("CapacityRemaining"),
- jsonObject.getLong("CapacityTotal"),
- jsonObject.getLong("FilesTotal"),
- jsonObject.getLong("BlocksTotal"),
- jsonObject.getLong("MissingBlocks"),
- jsonObject.getLong("PendingReplicationBlocks"),
- jsonObject.getLong("UnderReplicatedBlocks"),
- jsonObject.getLong("PendingDeletionBlocks"),
- jsonObject.optLong("ProvidedCapacityTotal"));
- }
- }
+ if (shouldUpdateJmx()) {
+ this.lastJmxUpdateAttempt = Time.monotonicNow();
+ String query = "Hadoop:service=NameNode,name=FSNamesystem*";
+ this.fsNamesystemMetrics = FederationUtil.getJmx(
+ query, address, connectionFactory, scheme);
}
+ populateFsNamesystemMetrics(this.fsNamesystemMetrics, report);
} catch (Exception e) {
LOG.error("Cannot get stat from {} using JMX", getNamenodeDesc(), e);
}
}
+ /**
+ * Evaluates whether the JMX report should be refreshed by
+ * calling the Namenode, based on the following conditions:
+ * 1. JMX Updates must be enabled.
+ * 2. The last attempt to update JMX occurred before the
+ * configured interval (if any).
+ */
+ private boolean shouldUpdateJmx() {
+ if (this.updateJmxIntervalMs < 0) {
+ return false;
+ }
+
+ return Time.monotonicNow() - this.lastJmxUpdateAttempt > this.updateJmxIntervalMs;
+ }
+
+ /**
+ * Populates FSNamesystem* metrics into report.
+ * @param aux FSNamesystem* metrics from namenode.
+ * @param report Namenode status report to update with JMX data.
+ * @throws JSONException When invalid JSONObject is found.
+ */
+ private void populateFsNamesystemMetrics(JSONArray aux, NamenodeStatusReport report)
+ throws JSONException {
+ if (aux != null) {
Review Comment:
Do early exit in == null and reduce nesting.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org