You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2021/02/01 12:42:51 UTC
[GitHub] [hadoop] tomscut opened a new pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock warnings
tomscut opened a new pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668
JIRA: https://issues.apache.org/jira/browse/HDFS-15808
To monitor how often read/write locks exceed thresholds, we can add two metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-782578219
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 18m 18s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 53s | | trunk passed |
| +1 :green_heart: | compile | 1m 20s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 15s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 22s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 17s | | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 58s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 28s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 3m 7s | | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 5s | | trunk passed |
| -0 :warning: | patch | 3m 24s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 16s | | the patch passed |
| +1 :green_heart: | compile | 1m 10s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 1m 6s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 1m 6s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 55s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 11s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 47s | | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 39s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | findbugs | 3m 57s | | the patch passed |
|||| _ Other Tests _ |
| -1 :x: | unit | 204m 49s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/6/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not generate ASF License warnings. |
| | | 312m 18s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| | hadoop.hdfs.qjournal.server.TestJournalNodeSync |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.datanode.TestBlockScanner |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2668 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 6ac55fde21d0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4891e68c2b2 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/6/testReport/ |
| Max. process+thread count | 3046 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/6/console |
| versions | git=2.25.1 maven=3.6.3 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-787609646
Hi @anuengineer , could you please review again? Let's see if there are any other questions. Thank you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut edited a comment on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut edited a comment on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780991248
I found this JIRA in the log. https://issues.apache.org/jira/browse/HADOOP-16870
Hi @aajisaka , Could you please help to take a look at this problem? And how I should deal with it? Thank you.
`[2021-02-17T01:59:03.362Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454)
[2021-02-17T01:59:03.887Z] Already on 'trunk'
[2021-02-17T01:59:03.887Z] Your branch is up to date with 'origin/trunk'.
[2021-02-17T01:59:04.412Z] Already up to date.
[2021-02-17T01:59:04.412Z] Current branch trunk is up to date.
[2021-02-17T01:59:04.412Z] Already on 'trunk'
[2021-02-17T01:59:04.412Z] Your branch is up to date with 'origin/trunk'.
[2021-02-17T01:59:05.444Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454)
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] Testing https://github.com/apache/hadoop/pull/2668 diff on trunk.
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] Re-exec mode detected. Continuing.
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] ERROR: Unprocessed flag(s): --findbugs-strict-precheck`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock warnings
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-771457006
Failed junit tests
hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
Sorry. I didn't change those two unit tests, and they worked fine locally.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on a change in pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut commented on a change in pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#discussion_r579912802
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystemLock.java
##########
@@ -182,6 +192,7 @@ public void readUnlock(String opName,
final long readLockIntervalMs =
TimeUnit.NANOSECONDS.toMillis(readLockIntervalNanos);
if (needReport && readLockIntervalMs >= this.readLockReportingThresholdMs) {
+ numReadLockLongHold.addAndGet(1);
Review comment:
> nit: you could use incrementAndGet. Don't have to fix it for me to commit it, just flagging for your consideration.
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/AtomicLong.html#incrementAndGet--
Hi @anuengineer , I fixed the problem. And I invited you to join the discussion on JIRA.
https://issues.apache.org/jira/browse/HDFS-15808
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock warnings
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-775605267
@xkrogen
Hi Erik Krogen, could you please review the code? Thanks a lot!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut removed a comment on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut removed a comment on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780991248
I found this JIRA in the log. https://issues.apache.org/jira/browse/HADOOP-16870
Hi @aajisaka , Could you please help to take a look at this problem? And how I should deal with it? Thank you.
`[2021-02-17T01:59:03.362Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454)
[2021-02-17T01:59:03.887Z] Already on 'trunk'
[2021-02-17T01:59:03.887Z] Your branch is up to date with 'origin/trunk'.
[2021-02-17T01:59:04.412Z] Already up to date.
[2021-02-17T01:59:04.412Z] Current branch trunk is up to date.
[2021-02-17T01:59:04.412Z] Already on 'trunk'
[2021-02-17T01:59:04.412Z] Your branch is up to date with 'origin/trunk'.
[2021-02-17T01:59:05.444Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454)
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] Testing https://github.com/apache/hadoop/pull/2668 diff on trunk.
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] Re-exec mode detected. Continuing.
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] ERROR: Unprocessed flag(s): --findbugs-strict-precheck`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-778256569
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 14m 56s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 7s | | trunk passed |
| +1 :green_heart: | compile | 1m 20s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 14s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 20s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 32s | | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 56s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 25s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 3m 5s | | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 3s | | trunk passed |
| -0 :warning: | patch | 3m 21s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 1m 11s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 11s | | the patch passed |
| +1 :green_heart: | compile | 1m 6s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 1m 6s | | the patch passed |
| -0 :warning: | checkstyle | 0m 57s | [/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/3/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs-project/hadoop-hdfs: The patch generated 8 new + 167 unchanged - 0 fixed = 175 total (was 167) |
| +1 :green_heart: | mvnsite | 1m 11s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 12m 55s | | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 48s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 17s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | findbugs | 3m 2s | | the patch passed |
|||| _ Other Tests _ |
| -1 :x: | unit | 190m 34s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 43s | | The patch does not generate ASF License warnings. |
| | | 289m 36s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2668 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 15073fe6c019 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f9a073c6c18 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/3/testReport/ |
| Max. process+thread count | 3120 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/3/console |
| versions | git=2.25.1 maven=3.6.3 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock warnings
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-771457006
Failed junit tests
hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
Sorry. I didn't change those two unit tests, and they worked fine locally.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut edited a comment on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut edited a comment on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780991248
I found this JIRA in the log. https://issues.apache.org/jira/browse/HADOOP-16870
Hi @aajisaka , Could you please help to take a look at this problem? And how I should deal with this problem? Thank you.
`[2021-02-17T01:59:03.362Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454)
[2021-02-17T01:59:03.887Z] Already on 'trunk'
[2021-02-17T01:59:03.887Z] Your branch is up to date with 'origin/trunk'.
[2021-02-17T01:59:04.412Z] Already up to date.
[2021-02-17T01:59:04.412Z] Current branch trunk is up to date.
[2021-02-17T01:59:04.412Z] Already on 'trunk'
[2021-02-17T01:59:04.412Z] Your branch is up to date with 'origin/trunk'.
[2021-02-17T01:59:05.444Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454)
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] Testing https://github.com/apache/hadoop/pull/2668 diff on trunk.
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] Re-exec mode detected. Continuing.
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] ERROR: Unprocessed flag(s): --findbugs-strict-precheck`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on a change in pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut commented on a change in pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#discussion_r579912802
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystemLock.java
##########
@@ -182,6 +192,7 @@ public void readUnlock(String opName,
final long readLockIntervalMs =
TimeUnit.NANOSECONDS.toMillis(readLockIntervalNanos);
if (needReport && readLockIntervalMs >= this.readLockReportingThresholdMs) {
+ numReadLockLongHold.addAndGet(1);
Review comment:
> nit: you could use incrementAndGet. Don't have to fix it for me to commit it, just flagging for your consideration.
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/AtomicLong.html#incrementAndGet--
Hi @anuengineer , I fixed the problem. And I invited you to join the discussion on JIRA.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut edited a comment on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock warnings
Posted by GitBox <gi...@apache.org>.
tomscut edited a comment on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-771457006
Failed junit tests
hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
Sorry. I didn't update those two unit tests, and they worked fine locally.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-790235902
Hi @shvachko , I uploaded a patch for branch-3.3 in JIRA, please help to review it. Thank you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-779330421
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 39s | | trunk passed |
| +1 :green_heart: | compile | 1m 20s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 13s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 3s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 19s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 56s | | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 27s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 3m 6s | | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 4s | | trunk passed |
| -0 :warning: | patch | 3m 22s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 16s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 16s | | the patch passed |
| +1 :green_heart: | compile | 1m 7s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 1m 7s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 58s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 14s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 13m 4s | | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 50s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 20s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | findbugs | 3m 9s | | the patch passed |
|||| _ Other Tests _ |
| -1 :x: | unit | 194m 59s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 44s | | The patch does not generate ASF License warnings. |
| | | 281m 25s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.namenode.ha.TestBootstrapAliasmap |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2668 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 0c1c25c99215 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c3134ab3a99 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/4/testReport/ |
| Max. process+thread count | 3175 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/4/console |
| versions | git=2.25.1 maven=3.6.3 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-797884140
Close this PR because the associated patch has been merged.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut closed pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut closed pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-783798764
Failed junit tests
hadoop.hdfs.server.namenode.ha.TestHAAppend
hadoop.hdfs.qjournal.server.TestJournalNodeSync
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
hadoop.hdfs.server.datanode.TestBlockScanner
Those failed unit tests were unrelated to the change. And they worked fine locally.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut removed a comment on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut removed a comment on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780595016
> This message was automatically generated.
Hi @anuengineer, the failed check looks unrelated to the change. May I ask how I should deal with it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] shvachko edited a comment on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
shvachko edited a comment on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-781609671
Just reposting my comment from the jira for visibility.
The patch looks fine, but I doubt the metric will be useful in its current form. Monotonically increasing counter doesn't tell you much when plotted. Over time it just becomes an incredibly large number, hard to see its fluctuations. And you cannot set alerts if the threshold is exceeded often.
See e.g. ExpiredHeartbeats or LastWrittenTransactionId - not useful.
I assume you need something like a rate.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-789484964
Hi @shvachko , I uploaded a patch for branch-3.3 in JIRA.
> +1 LGTM
Thanks @dineshchitlangia for the review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] anuengineer commented on a change in pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
anuengineer commented on a change in pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#discussion_r576928327
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystemLock.java
##########
@@ -182,6 +192,7 @@ public void readUnlock(String opName,
final long readLockIntervalMs =
TimeUnit.NANOSECONDS.toMillis(readLockIntervalNanos);
if (needReport && readLockIntervalMs >= this.readLockReportingThresholdMs) {
+ numReadLockLongHold.addAndGet(1);
Review comment:
nit: you could use incrementAndGet. Don't have to fix it for me to commit it, just flagging for your consideration.
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/AtomicLong.html#incrementAndGet--
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-788061149
Hi @arp7 , could you take a quick look at this? Let's see if there are any other questions. Thank you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780991248
I found this JIRA in the log. https://issues.apache.org/jira/browse/HADOOP-16870
Hi @aajisaka , Could you please help to take a look at this problem? And how I should deal with this problem? Thank you.
`[2021-02-17T01:59:03.362Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454)
[2021-02-17T01:59:03.887Z] Already on 'trunk'
[2021-02-17T01:59:03.887Z] Your branch is up to date with 'origin/trunk'.
[2021-02-17T01:59:04.412Z] Already up to date.
[2021-02-17T01:59:04.412Z] Current branch trunk is up to date.
[2021-02-17T01:59:04.412Z] Already on 'trunk'
[2021-02-17T01:59:04.412Z] Your branch is up to date with 'origin/trunk'.
[2021-02-17T01:59:05.444Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454)
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] Testing https://github.com/apache/hadoop/pull/2668 diff on trunk.
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] Re-exec mode detected. Continuing.
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] ERROR: Unprocessed flag(s): --findbugs-strict-precheck`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780595016
> This message was automatically generated.
Hi @anuengineer, the failed check looks unrelated to the change. May I ask how I should deal with it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-779345021
Failed junit tests
hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
hadoop.hdfs.server.namenode.ha.TestBootstrapAliasmap
Those failed unit tests were unrelated to the change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780242371
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 19m 14s | | Docker mode activated. |
| -1 :x: | yetus | 0m 6s | | Unprocessed flag(s): --findbugs-strict-precheck |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2668 |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/5/console |
| versions | git=2.25.1 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-778526792
Failed junit tests | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
Sorry. I didn't update this unit tests, and they worked fine locally.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut edited a comment on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock warnings
Posted by GitBox <gi...@apache.org>.
tomscut edited a comment on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-771457006
Failed junit tests
hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
Sorry. I didn't update those two unit tests, and they worked fine locally.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-781760195
> Just reposting my comment from the jira for visibility.
>
> The patch looks fine, but I doubt the metric will be useful in its current form. Monotonically increasing counter doesn't tell you much when plotted. Over time it just becomes an incredibly large number, hard to see its fluctuations. And you cannot set alerts if the threshold is exceeded often.
> See e.g. ExpiredHeartbeats or LastWrittenTransactionId - not useful.
> I assume you need something like a rate.
Hey @shvachko , thank you for your comments and suggestions. I replied to you in JIRA.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock warnings
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-771020821
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 52s | | trunk passed |
| +1 :green_heart: | compile | 1m 22s | | trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 17s | | trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| +1 :green_heart: | checkstyle | 1m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 23s | | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 58s | | trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 26s | | trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| +0 :ok: | spotbugs | 3m 19s | | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 17s | | trunk passed |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 11s | | the patch passed |
| +1 :green_heart: | compile | 1m 15s | | the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 1m 12s | | the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| +1 :green_heart: | javac | 1m 12s | | the patch passed |
| -0 :warning: | checkstyle | 1m 1s | [/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/1/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs-project/hadoop-hdfs: The patch generated 8 new + 165 unchanged - 0 fixed = 173 total (was 165) |
| +1 :green_heart: | mvnsite | 1m 12s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 13m 21s | | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 48s | | the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 27s | | the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| +1 :green_heart: | findbugs | 3m 18s | | the patch passed |
|||| _ Other Tests _ |
| -1 :x: | unit | 193m 50s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 41s | | The patch does not generate ASF License warnings. |
| | | 281m 31s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2668 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux a123314fa64f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 115623a6ee4 |
| Default Java | Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/1/testReport/ |
| Max. process+thread count | 3065 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/1/console |
| versions | git=2.25.1 maven=3.6.3 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut edited a comment on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut edited a comment on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780991248
I found this JIRA in the log. https://issues.apache.org/jira/browse/HADOOP-16870
Hi @aajisaka , Could you please help to take a look at this problem? And how I should deal with this problem? Thank you.
`[2021-02-17T01:59:03.362Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454)
[2021-02-17T01:59:03.887Z] Already on 'trunk'
[2021-02-17T01:59:03.887Z] Your branch is up to date with 'origin/trunk'.
[2021-02-17T01:59:04.412Z] Already up to date.
[2021-02-17T01:59:04.412Z] Current branch trunk is up to date.
[2021-02-17T01:59:04.412Z] Already on 'trunk'
[2021-02-17T01:59:04.412Z] Your branch is up to date with 'origin/trunk'.
[2021-02-17T01:59:05.444Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454)
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] Testing https://github.com/apache/hadoop/pull/2668 diff on trunk.
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] Re-exec mode detected. Continuing.
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] ERROR: Unprocessed flag(s): --findbugs-strict-precheck`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on a change in pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut commented on a change in pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#discussion_r577266767
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystemLock.java
##########
@@ -182,6 +192,7 @@ public void readUnlock(String opName,
final long readLockIntervalMs =
TimeUnit.NANOSECONDS.toMillis(readLockIntervalNanos);
if (needReport && readLockIntervalMs >= this.readLockReportingThresholdMs) {
+ numReadLockLongHold.addAndGet(1);
Review comment:
Thank you for your review and suggestion. Using incrementAndGet is a better way. I fixed it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut edited a comment on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut edited a comment on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-780991248
I found this JIRA in the log. https://issues.apache.org/jira/browse/HADOOP-16870
Hi @aajisaka , Could you please help to take a look at this problem? And how I should deal with this problem? Thank you.
`[2021-02-17T01:59:03.362Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454)
[2021-02-17T01:59:03.887Z] Already on 'trunk'
[2021-02-17T01:59:03.887Z] Your branch is up to date with 'origin/trunk'.
[2021-02-17T01:59:04.412Z] Already up to date.
[2021-02-17T01:59:04.412Z] Current branch trunk is up to date.
[2021-02-17T01:59:04.412Z] Already on 'trunk'
[2021-02-17T01:59:04.412Z] Your branch is up to date with 'origin/trunk'.
[2021-02-17T01:59:05.444Z] HEAD is now at 4cf35315838 HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454)
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] Testing https://github.com/apache/hadoop/pull/2668 diff on trunk.
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] Re-exec mode detected. Continuing.
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z] ============================================================================
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z]
[2021-02-17T01:59:05.444Z] ERROR: Unprocessed flag(s): --findbugs-strict-precheck`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] shvachko commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
shvachko commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-781609671
Just reposting my comment on the jira for visibility.
The patch looks fine, but I doubt the metric will be useful in its current form. Monotonically increasing counter doesn't tell you much when plotted. Over time it just becomes an incredibly large number, hard to see its fluctuations. And you cannot set alerts if the threshold is exceeded often.
See e.g. ExpiredHeartbeats or LastWrittenTransactionId - not useful.
I assume you need something like a rate.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-778248179
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 16s | | trunk passed |
| +1 :green_heart: | compile | 1m 25s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 17s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 35s | | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 54s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 24s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 3m 8s | | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 6s | | trunk passed |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 13s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 13s | | the patch passed |
| +1 :green_heart: | compile | 1m 5s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 1m 5s | | the patch passed |
| -0 :warning: | checkstyle | 0m 57s | [/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/2/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs-project/hadoop-hdfs: The patch generated 8 new + 167 unchanged - 0 fixed = 175 total (was 167) |
| +1 :green_heart: | mvnsite | 1m 12s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 13m 7s | | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 54s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 25s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | findbugs | 3m 12s | | the patch passed |
|||| _ Other Tests _ |
| -1 :x: | unit | 192m 53s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not generate ASF License warnings. |
| | | 280m 44s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized |
| | hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2668 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 4e47b5746c6d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f9a073c6c18 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/2/testReport/ |
| Max. process+thread count | 3398 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/2/console |
| versions | git=2.25.1 maven=3.6.3 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut edited a comment on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
tomscut edited a comment on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-779345021
Failed junit tests
hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
hadoop.hdfs.server.namenode.ha.TestBootstrapAliasmap
Those failed unit tests are unrelated to the change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] anuengineer commented on a change in pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time
Posted by GitBox <gi...@apache.org>.
anuengineer commented on a change in pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#discussion_r576929632
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystemLock.java
##########
@@ -182,6 +192,7 @@ public void readUnlock(String opName,
final long readLockIntervalMs =
TimeUnit.NANOSECONDS.toMillis(readLockIntervalNanos);
if (needReport && readLockIntervalMs >= this.readLockReportingThresholdMs) {
+ numReadLockLongHold.addAndGet(1);
Review comment:
+1, LGTM. if you are making any changes, please let me know. Otherwise, I will leave for a little while for @arp7 or @xiaoyuyao to take a quick look. Thank you for your work and contribution.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tomscut edited a comment on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock warnings
Posted by GitBox <gi...@apache.org>.
tomscut edited a comment on pull request #2668:
URL: https://github.com/apache/hadoop/pull/2668#issuecomment-775605267
@xkrogen
Hi Erik, could you please review the code? Thanks a lot!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org