You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2021/09/23 12:24:41 UTC
[GitHub] [hadoop] haiyang1987 opened a new pull request #3471: HDFS-16231. Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
haiyang1987 opened a new pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471
Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
https://issues.apache.org/jira/browse/HDFS-16231
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ferhui commented on a change in pull request #3471: HDFS-16231. Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
Posted by GitBox <gi...@apache.org>.
ferhui commented on a change in pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471#discussion_r716131815
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMetrics.java
##########
@@ -190,18 +190,28 @@ public Object answer(InvocationOnMock invocationOnMock)
DataNodeFaultInjector.set(injector);
Review comment:
And here we need to reserve original injector and reset it at finally scope.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] haiyang1987 edited a comment on pull request #3471: HDFS-16231. Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
Posted by GitBox <gi...@apache.org>.
haiyang1987 edited a comment on pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471#issuecomment-926538825
@ferhui Thanks for your reply！
There are two problems:
1.Current code error MetricsName, unable to get actual metrics，e.g TotalPacketsReceived，TotalPacketsSlowWriteToMirror，TotalPacketsSlowWriteToDisk，TotalPacketsSlowWriteToOsCache
MetricsRecordBuilder dnMetrics = getMetrics(datanode.getMetrics().name());
assertTrue("More than 1 packet received",
getLongCounter("TotalPacketsReceived", dnMetrics) > 1L);
assertTrue("More than 1 slow packet to mirror",
getLongCounter("TotalPacketsSlowWriteToMirror", dnMetrics) > 1L);
assertCounter("TotalPacketsSlowWriteToDisk", 1L, dnMetrics);
assertCounter("TotalPacketsSlowWriteToOsCache", 0L, dnMetrics);
2.Current, we need to get the first or second DataNode of the pipline that writes data to get ReceivePacketSlowMetrics of the DataNode
```
List<DataNode> datanodes = cluster.getDataNodes();
DataNode datanode = datanodes.get(0);
//If the datanode obtained here is the third node of the pipline to write data, such as PacketsSlowWriteToMirror unable to get
MetricsRecordBuilder dnMetrics = getMetrics(datanode.getMetrics().name());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] haiyang1987 edited a comment on pull request #3471: HDFS-16231. Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
Posted by GitBox <gi...@apache.org>.
haiyang1987 edited a comment on pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471#issuecomment-926538825
@ferhui Thanks for your reply！
There are two problems:
1.Current code error MetricsName, unable to get actual metrics，e.g TotalPacketsReceived，TotalPacketsSlowWriteToMirror，TotalPacketsSlowWriteToDisk，TotalPacketsSlowWriteToOsCache
MetricsRecordBuilder dnMetrics = getMetrics(datanode.getMetrics().name());
assertTrue("More than 1 packet received",
getLongCounter("TotalPacketsReceived", dnMetrics) > 1L);
assertTrue("More than 1 slow packet to mirror",
getLongCounter("TotalPacketsSlowWriteToMirror", dnMetrics) > 1L);
assertCounter("TotalPacketsSlowWriteToDisk", 1L, dnMetrics);
assertCounter("TotalPacketsSlowWriteToOsCache", 0L, dnMetrics);
2.Current, we need to get the first or second DataNode of the pipline that writes data to get ReceivePacketSlowMetrics of the DataNode
```
List<DataNode> datanodes = cluster.getDataNodes();
DataNode datanode = datanodes.get(0);
//If the datanode obtained here is the third node of the pipline to write data,
such as PacketsSlowWriteToMirror unable to get
MetricsRecordBuilder dnMetrics = getMetrics(datanode.getMetrics().name());
...
assertTrue("More than 1 slow packet to mirror",
getLongCounter("PacketsSlowWriteToMirror", dnMetrics) > 1L);
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3471: HDFS-16231. Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471#issuecomment-926186033
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 2m 24s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 36m 19s | | trunk passed |
| +1 :green_heart: | compile | 1m 25s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 16s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 0s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 23s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 56s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 28s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 17s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 1m 20s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 20s | | the patch passed |
| +1 :green_heart: | compile | 1m 10s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 10s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 53s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 15s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 50s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 20s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 24s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 18s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 450m 19s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3471/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| -1 :x: | asflicense | 1m 7s | [/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3471/1/artifact/out/results-asflicense.txt) | The patch generated 7 ASF License warnings. |
| | | 558m 47s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes |
| | hadoop.hdfs.TestViewDistributedFileSystemWithMountLinks |
| | hadoop.hdfs.server.mover.TestMover |
| | hadoop.hdfs.TestErasureCodingPolicies |
| | hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks |
| | hadoop.hdfs.TestEncryptionZones |
| | hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy |
| | hadoop.hdfs.server.namenode.TestGetContentSummaryWithPermission |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3471/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/3471 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux c8c44c8fb51b 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 3cdc7d9f75e33182423a2b6b59d860c42708db32 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3471/1/testReport/ |
| Max. process+thread count | 2211 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3471/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ferhui merged pull request #3471: HDFS-16231. Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
Posted by GitBox <gi...@apache.org>.
ferhui merged pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] haiyang1987 commented on a change in pull request #3471: HDFS-16231. Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
Posted by GitBox <gi...@apache.org>.
haiyang1987 commented on a change in pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471#discussion_r716195494
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMetrics.java
##########
@@ -190,18 +190,28 @@ public Object answer(InvocationOnMock invocationOnMock)
DataNodeFaultInjector.set(injector);
Review comment:
hi @ferhui Thank you for your comments!
It was revised as you suggested
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] haiyang1987 commented on pull request #3471: HDFS-16231. Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
Posted by GitBox <gi...@apache.org>.
haiyang1987 commented on pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471#issuecomment-928867034
@ferhui Thanks your review and merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ferhui merged pull request #3471: HDFS-16231. Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
Posted by GitBox <gi...@apache.org>.
ferhui merged pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ferhui commented on pull request #3471: HDFS-16231. Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
Posted by GitBox <gi...@apache.org>.
ferhui commented on pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471#issuecomment-926295605
@haiyang1987 Could you please give more details?
Explain the failed reason and how you fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] haiyang1987 commented on pull request #3471: HDFS-16231. Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
Posted by GitBox <gi...@apache.org>.
haiyang1987 commented on pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471#issuecomment-928867034
@ferhui Thanks your review and merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ferhui commented on pull request #3471: HDFS-16231. Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
Posted by GitBox <gi...@apache.org>.
ferhui commented on pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471#issuecomment-928825711
@haiyang1987 Thanks for contribution. Merged
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ferhui commented on a change in pull request #3471: HDFS-16231. Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
Posted by GitBox <gi...@apache.org>.
ferhui commented on a change in pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471#discussion_r716051187
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMetrics.java
##########
@@ -190,18 +190,28 @@ public Object answer(InvocationOnMock invocationOnMock)
DataNodeFaultInjector.set(injector);
Path testFile = new Path("/testFlushNanosMetric.txt");
FSDataOutputStream fout = fs.create(testFile);
+ DFSOutputStream dout = (DFSOutputStream) fout.getWrappedStream();
fout.write(new byte[1]);
fout.hsync();
+ DatanodeInfo[] pipeline = dout.getPipeline();
fout.close();
+ dout.close();
+ DatanodeInfo headDatanodeInfo = pipeline[0];
+ DataNode headNode = null;
List<DataNode> datanodes = cluster.getDataNodes();
- DataNode datanode = datanodes.get(0);
- MetricsRecordBuilder dnMetrics = getMetrics(datanode.getMetrics().name());
+ for (DataNode datanode : datanodes) {
Review comment:
Could we change this to lambda like this
` DataNode headNode = datanodes.stream()
.filter(d -> d.getDatanodeId().equals(headDatanodeInfo))
.findFirst().orElseGet(() -> null);
`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3471: HDFS-16231. Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471#issuecomment-927358474
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 59s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 34s | | trunk passed |
| +1 :green_heart: | compile | 1m 25s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 15s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 59s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 21s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 56s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 26s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 15s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 0s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 14s | | the patch passed |
| +1 :green_heart: | compile | 1m 19s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 19s | | the patch passed |
| +1 :green_heart: | compile | 1m 9s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 9s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 54s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 14s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 48s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 23s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 8s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 346m 7s | | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not generate ASF License warnings. |
| | | 449m 50s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3471/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/3471 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 593cc9c1aa7d 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / bb1508d900136c08e12f80d5c2e037ee47df4eca |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3471/2/testReport/ |
| Max. process+thread count | 2036 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3471/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] haiyang1987 commented on pull request #3471: HDFS-16231. Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
Posted by GitBox <gi...@apache.org>.
haiyang1987 commented on pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471#issuecomment-926538825
@ferhui Thanks for your reply！
There are two problems:
1.Current code error MetricsName, unable to get actual metrics，e.g TotalPacketsReceived，TotalPacketsSlowWriteToMirror，TotalPacketsSlowWriteToDisk，TotalPacketsSlowWriteToOsCache
MetricsRecordBuilder dnMetrics = getMetrics(datanode.getMetrics().name());
assertTrue("More than 1 packet received",
getLongCounter("TotalPacketsReceived", dnMetrics) > 1L);
assertTrue("More than 1 slow packet to mirror",
getLongCounter("TotalPacketsSlowWriteToMirror", dnMetrics) > 1L);
assertCounter("TotalPacketsSlowWriteToDisk", 1L, dnMetrics);
assertCounter("TotalPacketsSlowWriteToOsCache", 0L, dnMetrics);
2.Current, we need to get the first or second DataNode of the pipline that writes data to get ReceivePacketSlowMetrics of the DataNode
```
List<DataNode> datanodes = cluster.getDataNodes();
DataNode datanode = datanodes.get(0);
//If the datanode obtained here is the third node of the pipline to write data, such as PacketsSlowWriteToMirror,PacketsSlowWriteToDisk,PacketsSlowWriteToOsCache unable to get
MetricsRecordBuilder dnMetrics = getMetrics(datanode.getMetrics().name());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ferhui commented on pull request #3471: HDFS-16231. Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
Posted by GitBox <gi...@apache.org>.
ferhui commented on pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471#issuecomment-928825711
@haiyang1987 Thanks for contribution. Merged
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org