You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2020/08/24 17:42:41 UTC
[GitHub] [hadoop] 1996fanrui opened a new pull request #2241: HADOOP-17222. Create socket address combined with cache
1996fanrui opened a new pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241
https://issues.apache.org/jira/browse/HADOOP-17222
Create socket address combined with cache to speed up hdfs client choose DataNode
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-689272438
> @1996fanrui could you try the above command shared by @dineshchitlangia to trigger a new PreCommit QA run? Hopefully it will be clean next time. Thanks,
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-681908227
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 3m 55s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 3m 20s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 34s | trunk passed |
| +1 :green_heart: | compile | 20m 8s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 17m 18s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 2m 47s | trunk passed |
| +1 :green_heart: | mvnsite | 4m 7s | trunk passed |
| +1 :green_heart: | shadedclient | 21m 26s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 35s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 4m 4s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 35s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 7m 59s | trunk passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 50s | the patch passed |
| +1 :green_heart: | compile | 18m 49s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 18m 49s | the patch passed |
| +1 :green_heart: | compile | 17m 34s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 17m 34s | the patch passed |
| +1 :green_heart: | checkstyle | 2m 43s | root: The patch generated 0 new + 136 unchanged - 2 fixed = 136 total (was 138) |
| +1 :green_heart: | mvnsite | 3m 58s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 16m 32s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 29s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 4m 12s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 9m 51s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 10m 50s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 2m 31s | hadoop-hdfs-client in the patch passed. |
| -1 :x: | unit | 146m 28s | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 15s | The patch does not generate ASF License warnings. |
| | | 353m 14s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.metrics2.source.TestJvmMetrics |
| | hadoop.hdfs.TestErasureCodingPolicyWithSnapshot |
| | hadoop.hdfs.TestFileChecksumCompositeCrc |
| | hadoop.hdfs.TestFileChecksum |
| | hadoop.hdfs.TestReconstructStripedFile |
| | hadoop.hdfs.server.balancer.TestBalancerRPCDelay |
| | hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader |
| | hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics |
| | hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes |
| | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier |
| | hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand |
| | hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/9/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2241 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml |
| uname | Linux 1629946657fc 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2ffe00fc46a |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| unit | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/9/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt |
| unit | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/9/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/9/testReport/ |
| Max. process+thread count | 4104 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/9/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r476391343
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/NetUtils.java
##########
@@ -218,6 +211,28 @@ public static InetSocketAddress createSocketAddr(String target,
return createSocketAddrForHost(host, port);
}
+ private static final Map<String, URI> URI_CACHE = new ConcurrentHashMap<>();
Review comment:
Hi @Hexiaoqiao
DFSUtilClient.isLocalAddress also uses Cache for acceleration. It is basically similar to the maximum data size of the current patch Cache, that is, the number of DNs. The Cache of DFSUtilClient.isLocalAddress does not use the evict policy, so the current patch should not worry about taking up too much memory.
Of course, if the evict policy is added, the code will be more robust.
If necessary, I can use guava Cache instead of Map. What do you think?
https://github.com/apache/hadoop/blob/82a750564631eb3077603bed27058006b3d66309/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSUtilClient.java#L634
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
liuml07 commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-690850909
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 merged pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
liuml07 merged pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
liuml07 commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-689153364
Was waiting for a clean QA run but could not trigger it manually. If I get no help from community, we can upload the patch file to the JIRA and from there I can trigger multiple runs to get a clean pre-commit build.
I assume failing tests are not related to this but it's always nice to confirm.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 edited a comment on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
liuml07 edited a comment on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-690850909
I see `TestNameNodeRetryCacheMetrics.testRetryCacheMetrics` and ` TestMultipleNNPortQOP.testMultipleNNPortOverwriteDownStream` fail on my Linux machine with the patch. Will confirm they are not related.
If no objections, I'll commit later today. Thanks,
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
liuml07 commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-690885275
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-681796610
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 36s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 3m 24s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 14s | trunk passed |
| +1 :green_heart: | compile | 20m 54s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 19m 26s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 3m 31s | trunk passed |
| +1 :green_heart: | mvnsite | 4m 22s | trunk passed |
| +1 :green_heart: | shadedclient | 25m 12s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 39s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 4m 32s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 3m 7s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 9m 19s | trunk passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 12s | the patch passed |
| +1 :green_heart: | compile | 23m 4s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 23m 4s | the patch passed |
| +1 :green_heart: | compile | 20m 47s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 20m 47s | the patch passed |
| +1 :green_heart: | checkstyle | 3m 18s | root: The patch generated 0 new + 136 unchanged - 2 fixed = 136 total (was 138) |
| +1 :green_heart: | mvnsite | 4m 10s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 14m 7s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 33s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 3m 58s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 8m 21s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 20s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 2m 21s | hadoop-hdfs-client in the patch passed. |
| -1 :x: | unit | 94m 38s | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 7s | The patch does not generate ASF License warnings. |
| | | 310m 38s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.TestFileChecksumCompositeCrc |
| | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped |
| | hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader |
| | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier |
| | hadoop.hdfs.TestFileChecksum |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/7/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2241 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml |
| uname | Linux 246df61e9427 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ca8e7a77256 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| unit | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/7/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/7/testReport/ |
| Max. process+thread count | 4210 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/7/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
liuml07 commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-690885275
Failing tests are not related and all pass locally in my laptop except `TestNameNodeRetryCacheMetrics` which is known flaky see [HDFS-15458](https://issues.apache.org/jira/browse/HDFS-15458)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] Hexiaoqiao commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
Hexiaoqiao commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r476127785
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/NetUtils.java
##########
@@ -218,6 +211,28 @@ public static InetSocketAddress createSocketAddr(String target,
return createSocketAddrForHost(host, port);
}
+ private static final Map<String, URI> URI_CACHE = new ConcurrentHashMap<>();
Review comment:
Thanks @1996fanrui for your proposal here.
A. Not sure if this improvement could impact some other modules because {{NetUtil}} is very common util for different modules rather than DFSClient only.
B. It is better to make the scope to DFSClient only if we could based on the original description [HADOOP-17222](https://issues.apache.org/jira/browse/HADOOP-17222).
C. add evict policy for {{URI_CACHE}}?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-681093538
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 3m 20s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 20s | trunk passed |
| +1 :green_heart: | compile | 21m 33s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 18m 54s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 2m 48s | trunk passed |
| +1 :green_heart: | mvnsite | 4m 17s | trunk passed |
| +1 :green_heart: | shadedclient | 21m 25s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 30s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 3m 59s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 45s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 8m 9s | trunk passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 51s | the patch passed |
| +1 :green_heart: | compile | 20m 56s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 20m 56s | the patch passed |
| +1 :green_heart: | compile | 18m 11s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 18m 11s | the patch passed |
| -0 :warning: | checkstyle | 2m 44s | root: The patch generated 1 new + 136 unchanged - 2 fixed = 137 total (was 138) |
| +1 :green_heart: | mvnsite | 3m 50s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 14m 29s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 21s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 3m 56s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 8m 52s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 35s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 2m 14s | hadoop-hdfs-client in the patch passed. |
| -1 :x: | unit | 113m 4s | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 9s | The patch does not generate ASF License warnings. |
| | | 316m 36s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.TestFileChecksumCompositeCrc |
| | hadoop.hdfs.TestDFSStorageStateRecovery |
| | hadoop.hdfs.server.balancer.TestBalancer |
| | hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot |
| | hadoop.hdfs.TestReconstructStripedFile |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication |
| | hadoop.hdfs.TestFileChecksum |
| | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier |
| | hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithRandomECPolicy |
| | hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| | hadoop.hdfs.server.namenode.TestReencryptionWithKMS |
| | hadoop.hdfs.server.blockmanagement.TestReconstructStripedBlocksWithRackAwareness |
| | hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks |
| | hadoop.hdfs.TestErasureCodeBenchmarkThroughput |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.blockmanagement.TestBlockInfoStriped |
| | hadoop.hdfs.server.namenode.TestNamenodeCapacityReport |
| | hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2241 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml |
| uname | Linux 4a4bc1b2b250 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 931adbaa141 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| checkstyle | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/6/artifact/out/diff-checkstyle-root.txt |
| unit | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/6/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/6/testReport/ |
| Max. process+thread count | 3629 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/6/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-681879637
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 3m 19s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 15s | trunk passed |
| +1 :green_heart: | compile | 19m 24s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 16m 48s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 2m 45s | trunk passed |
| +1 :green_heart: | mvnsite | 4m 8s | trunk passed |
| +1 :green_heart: | shadedclient | 21m 30s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 35s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 4m 5s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 35s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 7m 55s | trunk passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 51s | the patch passed |
| +1 :green_heart: | compile | 18m 47s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 18m 47s | the patch passed |
| +1 :green_heart: | compile | 16m 49s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 16m 49s | the patch passed |
| +1 :green_heart: | checkstyle | 2m 46s | root: The patch generated 0 new + 136 unchanged - 2 fixed = 136 total (was 138) |
| +1 :green_heart: | mvnsite | 4m 8s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 14m 4s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 34s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 4m 9s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 8m 20s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 48s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 2m 18s | hadoop-hdfs-client in the patch passed. |
| -1 :x: | unit | 96m 54s | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 4s | The patch does not generate ASF License warnings. |
| | | 292m 58s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| | hadoop.hdfs.TestFileChecksum |
| | hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader |
| | hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped |
| | hadoop.hdfs.TestFileChecksumCompositeCrc |
| | hadoop.hdfs.server.datanode.TestBPOfferService |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/8/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2241 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml |
| uname | Linux 382b70e7cecd 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2ffe00fc46a |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| unit | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/8/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/8/testReport/ |
| Max. process+thread count | 4341 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/8/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-681636223
> Looks good to me. Will leave to 2nd reviewer to commit. Thanks!
Thanks for your review. @liuml07
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 edited a comment on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
liuml07 edited a comment on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-690850909
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 merged pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
liuml07 merged pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
liuml07 commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r478114810
##########
File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java
##########
@@ -414,6 +415,11 @@
String PREFETCH_SIZE_KEY = PREFIX + "prefetch.size";
+ String URI_CACHE_KEY = PREFIX + "uri.cache.enable";
+ boolean URI_CACHE_DEFAULT = false;
+ String URI_CACHE_EXPIRE_MS_KEY = PREFIX + "uri.cache.expire.ms";
Review comment:
I think it's better not to have this config as I suggested in the JIRA. It's for two reasons:
1. This seems not critical parameter and Hadoop already has too many configurations. I think the default value 12hours is good enough. It's not meant to be tuned by per client since the cache is static and shared. So removing this config seems preferred to me.
2. The URI cache is initialized only once. So passing the `uriCacheExpireMs` parameter every time does not really carry useful information. And it may be confusing if people mistakenly interpret this as a per-URI setting when they call `NetUtils.createSocketAddr()` which is not.
Thoughts?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
liuml07 commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-690850909
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-688674895
> I'm now on vacation. I will merge it this week.
ok, thank you very much.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r478166422
##########
File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java
##########
@@ -414,6 +415,11 @@
String PREFETCH_SIZE_KEY = PREFIX + "prefetch.size";
+ String URI_CACHE_KEY = PREFIX + "uri.cache.enable";
+ boolean URI_CACHE_DEFAULT = false;
+ String URI_CACHE_EXPIRE_MS_KEY = PREFIX + "uri.cache.expire.ms";
Review comment:
Thanks for your review, I have fixed all the comments.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-680221389
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 29m 8s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 3m 18s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 46s | trunk passed |
| +1 :green_heart: | compile | 22m 50s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 20m 23s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 3m 1s | trunk passed |
| +1 :green_heart: | mvnsite | 4m 22s | trunk passed |
| +1 :green_heart: | shadedclient | 23m 19s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 21s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 3m 54s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 58s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 9m 3s | trunk passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 5s | the patch passed |
| +1 :green_heart: | compile | 23m 55s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 23m 55s | the patch passed |
| +1 :green_heart: | compile | 19m 57s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 19m 57s | the patch passed |
| -0 :warning: | checkstyle | 2m 58s | root: The patch generated 1 new + 114 unchanged - 2 fixed = 115 total (was 116) |
| +1 :green_heart: | mvnsite | 4m 5s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 16m 5s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 22s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 3m 54s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 9m 27s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 25s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 2m 23s | hadoop-hdfs-client in the patch passed. |
| -1 :x: | unit | 103m 51s | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 7s | The patch does not generate ASF License warnings. |
| | | 350m 25s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.TestSafeModeWithStripedFileWithRandomECPolicy |
| | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier |
| | hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader |
| | hadoop.hdfs.TestPread |
| | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithRandomECPolicy |
| | hadoop.hdfs.server.datanode.TestBPOfferService |
| | hadoop.tools.TestHdfsConfigFields |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2241 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml |
| uname | Linux b9d237abe091 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 82a75056463 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| checkstyle | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/5/artifact/out/diff-checkstyle-root.txt |
| unit | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/5/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/5/testReport/ |
| Max. process+thread count | 4219 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/5/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
liuml07 commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r478114810
##########
File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java
##########
@@ -414,6 +415,11 @@
String PREFETCH_SIZE_KEY = PREFIX + "prefetch.size";
+ String URI_CACHE_KEY = PREFIX + "uri.cache.enable";
+ boolean URI_CACHE_DEFAULT = false;
+ String URI_CACHE_EXPIRE_MS_KEY = PREFIX + "uri.cache.expire.ms";
Review comment:
I think it's better not to have this config as I suggested in the JIRA. It's for two reasons:
1. This seems not critical parameter and Hadoop already has too many configurations. I think the default value 12hours is good enough. It's not meant to be tuned by per client since the cache is static and shared. So removing this config seems preferred to me.
2. The URI cache is initialized only once. So passing the `uriCacheExpireMs` parameter overtime does not really carry enough information. And it may be confusing if people mistakenly interpret this as a per-URI setting when they call `NetUtils.createSocketAddr()` which is not.
Thoughts?
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
##########
@@ -4185,6 +4185,16 @@
</description>
</property>
+<property>
+ <name>dfs.client.read.uri.cache.enable</name>
+ <value>false</value>
+ <description>
+ If true, dfs client will use cache when creating URI based on host:port
+ to reduce the frequency of URI object creation.
+ Default is false.
Review comment:
nit: let's remove this sentence. The default value is clear in 5 lines above. Otherwise if we update the default value in future, we may miss this one which will cause confusion.
##########
File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java
##########
@@ -414,6 +414,9 @@
String PREFETCH_SIZE_KEY = PREFIX + "prefetch.size";
+ String URI_CACHE_KEY = PREFIX + "uri.cache.enable";
Review comment:
nit: name this `...uri.cache.enabled` by replacing `enable` with `enabled`?
Also change other variables related to `uriCacheEnable`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] Hexiaoqiao commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
Hexiaoqiao commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r478402307
##########
File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/DfsClientConf.java
##########
@@ -211,24 +212,7 @@ public DfsClientConf(Configuration conf) {
Write.MAX_PACKETS_IN_FLIGHT_KEY,
Write.MAX_PACKETS_IN_FLIGHT_DEFAULT);
- final boolean byteArrayManagerEnabled = conf.getBoolean(
- Write.ByteArrayManager.ENABLED_KEY,
- Write.ByteArrayManager.ENABLED_DEFAULT);
- if (!byteArrayManagerEnabled) {
- writeByteArrayManagerConf = null;
- } else {
- final int countThreshold = conf.getInt(
- Write.ByteArrayManager.COUNT_THRESHOLD_KEY,
- Write.ByteArrayManager.COUNT_THRESHOLD_DEFAULT);
- final int countLimit = conf.getInt(
- Write.ByteArrayManager.COUNT_LIMIT_KEY,
- Write.ByteArrayManager.COUNT_LIMIT_DEFAULT);
- final long countResetTimePeriodMs = conf.getLong(
- Write.ByteArrayManager.COUNT_RESET_TIME_PERIOD_MS_KEY,
- Write.ByteArrayManager.COUNT_RESET_TIME_PERIOD_MS_DEFAULT);
- writeByteArrayManagerConf = new ByteArrayManager.Conf(
- countThreshold, countLimit, countResetTimePeriodMs);
- }
+ writeByteArrayManagerConf = loadWriteByteArrayManagerConf(conf);
Review comment:
Thanks for your quick response, this is reasonable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-688160009
Hi @liuml07 ,Could you have time to help merge it? Thank you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] dineshchitlangia commented on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
dineshchitlangia commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-689158906
@liuml07 You could do an empty commit on the PR branch and push it.
`git commit --allow-empty -m 'trigger new CI check' && git push`
This should trigger CI
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-690855787
> I see `TestNameNodeRetryCacheMetrics.testRetryCacheMetrics` and ` TestMultipleNNPortQOP.testMultipleNNPortOverwriteDownStream` fail on my Linux machine with the patch. Will confirm they are not related.
>
> If no objections, I'll commit later this week. Thanks,
@liuml07 , thanks for your test.
I run these two unit tests and they are still successful. In the previous test of hadoop robot, TestMultipleNNPortQOP did not fail, only the last time it failed. But the last time I only committed an empty, my code did not change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-679386091
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 39s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 17s | trunk passed |
| +1 :green_heart: | compile | 24m 49s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 19m 59s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 46s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 25s | trunk passed |
| +1 :green_heart: | shadedclient | 17m 57s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 29s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 22s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 21s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 19s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 53s | the patch passed |
| +1 :green_heart: | compile | 20m 56s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 20m 56s | the patch passed |
| +1 :green_heart: | compile | 17m 31s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 17m 31s | the patch passed |
| -0 :warning: | checkstyle | 0m 45s | hadoop-common-project/hadoop-common: The patch generated 7 new + 33 unchanged - 0 fixed = 40 total (was 33) |
| +1 :green_heart: | mvnsite | 1m 22s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 40s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 27s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 23s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 2m 20s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 9m 22s | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 45s | The patch does not generate ASF License warnings. |
| | | 176m 29s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.security.TestSecurityUtil |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2241 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux f427c6771114 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 15a0fed6371 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| checkstyle | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/1/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt |
| unit | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/1/testReport/ |
| Max. process+thread count | 2472 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/1/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-682167781
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 29s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 3m 21s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 4s | trunk passed |
| +1 :green_heart: | compile | 19m 28s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 16m 47s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 2m 48s | trunk passed |
| +1 :green_heart: | mvnsite | 4m 6s | trunk passed |
| +1 :green_heart: | shadedclient | 20m 34s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 42s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 4m 6s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 3m 13s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 10m 10s | trunk passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 36s | the patch passed |
| +1 :green_heart: | compile | 25m 7s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 25m 7s | the patch passed |
| +1 :green_heart: | compile | 23m 1s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 23m 1s | the patch passed |
| +1 :green_heart: | checkstyle | 3m 23s | root: The patch generated 0 new + 136 unchanged - 2 fixed = 136 total (was 138) |
| +1 :green_heart: | mvnsite | 4m 32s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 18m 15s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 37s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 4m 27s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 10m 47s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 30s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 2m 41s | hadoop-hdfs-client in the patch passed. |
| -1 :x: | unit | 121m 36s | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 7s | The patch does not generate ASF License warnings. |
| | | 343m 16s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.TestErasureCodingPoliciesWithRandomECPolicy |
| | hadoop.hdfs.TestDFSClientSocketSize |
| | hadoop.hdfs.TestFileChecksumCompositeCrc |
| | hadoop.hdfs.TestDecommissionWithBackoffMonitor |
| | hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics |
| | hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints |
| | hadoop.hdfs.TestReadStripedFileWithDNFailure |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped |
| | hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy |
| | hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithRandomECPolicy |
| | hadoop.hdfs.TestAppendSnapshotTruncate |
| | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier |
| | hadoop.hdfs.TestReadStripedFileWithMissingBlocks |
| | hadoop.hdfs.TestFileChecksum |
| | hadoop.hdfs.TestAbandonBlock |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/10/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2241 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml |
| uname | Linux 08b2a55dd876 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d1c60a53f60 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| unit | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/10/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/10/testReport/ |
| Max. process+thread count | 4298 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/10/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-679827027
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 44s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 35m 2s | trunk passed |
| +1 :green_heart: | compile | 28m 13s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 21m 53s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 46s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 23s | trunk passed |
| +1 :green_heart: | shadedclient | 18m 10s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 30s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 24s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 13s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 11s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 49s | the patch passed |
| +1 :green_heart: | compile | 20m 10s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 20m 10s | the patch passed |
| +1 :green_heart: | compile | 17m 31s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 17m 31s | the patch passed |
| -0 :warning: | checkstyle | 0m 45s | hadoop-common-project/hadoop-common: The patch generated 2 new + 32 unchanged - 1 fixed = 34 total (was 33) |
| +1 :green_heart: | mvnsite | 1m 21s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 24s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 29s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 25s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 2m 21s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 50s | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 47s | The patch does not generate ASF License warnings. |
| | | 184m 25s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2241 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux bca38132d7d2 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 64f36b9543c |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| checkstyle | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/3/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/3/testReport/ |
| Max. process+thread count | 3233 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/3/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-690855787
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-679762667
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 27s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 29m 47s | trunk passed |
| +1 :green_heart: | compile | 21m 13s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 17m 48s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 47s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 30s | trunk passed |
| +1 :green_heart: | shadedclient | 17m 5s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 34s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 31s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 23s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 21s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 57s | the patch passed |
| +1 :green_heart: | compile | 20m 21s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 20m 21s | the patch passed |
| +1 :green_heart: | compile | 17m 44s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 17m 44s | the patch passed |
| -0 :warning: | checkstyle | 0m 46s | hadoop-common-project/hadoop-common: The patch generated 2 new + 32 unchanged - 1 fixed = 34 total (was 33) |
| +1 :green_heart: | mvnsite | 1m 36s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 13m 54s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 29s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 31s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 2m 26s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 32s | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 51s | The patch does not generate ASF License warnings. |
| | | 165m 27s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2241 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux ae8a45da1636 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 64f36b9543c |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| checkstyle | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/2/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/2/testReport/ |
| Max. process+thread count | 1430 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/2/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 edited a comment on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
liuml07 edited a comment on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-690850909
I see `TestNameNodeRetryCacheMetrics.testRetryCacheMetrics` and ` TestMultipleNNPortQOP.testMultipleNNPortOverwriteDownStream` fail on my Linux machine with the patch. Will confirm they are not related.
If no objections, I'll commit later today. Thanks,
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-689407545
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 29m 17s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 3m 25s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 32m 31s | trunk passed |
| +1 :green_heart: | compile | 26m 27s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 21m 48s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 3m 28s | trunk passed |
| +1 :green_heart: | mvnsite | 4m 31s | trunk passed |
| +1 :green_heart: | shadedclient | 26m 14s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 30s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 4m 21s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 3m 23s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 9m 49s | trunk passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 22s | the patch passed |
| +1 :green_heart: | compile | 19m 13s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 19m 13s | the patch passed |
| +1 :green_heart: | compile | 16m 48s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 16m 48s | the patch passed |
| +1 :green_heart: | checkstyle | 2m 42s | root: The patch generated 0 new + 136 unchanged - 2 fixed = 136 total (was 138) |
| +1 :green_heart: | mvnsite | 4m 3s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 14m 33s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 21s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 3m 53s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 8m 25s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 30s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 2m 18s | hadoop-hdfs-client in the patch passed. |
| -1 :x: | unit | 105m 20s | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 21s | The patch does not generate ASF License warnings. |
| | | 355m 56s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.TestGetFileChecksum |
| | hadoop.hdfs.TestFileChecksumCompositeCrc |
| | hadoop.hdfs.server.namenode.ha.TestHASafeMode |
| | hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics |
| | hadoop.hdfs.TestMultipleNNPortQOP |
| | hadoop.hdfs.server.namenode.TestAddStripedBlocks |
| | hadoop.hdfs.server.namenode.TestNamenodeRetryCache |
| | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.namenode.TestNameEditsConfigs |
| | hadoop.hdfs.TestFileChecksum |
| | hadoop.hdfs.server.namenode.TestFsck |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/11/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2241 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml |
| uname | Linux 88a54f877166 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0d855159f09 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| unit | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/11/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/11/testReport/ |
| Max. process+thread count | 4040 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2241/11/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 merged pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
liuml07 merged pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r478397859
##########
File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestNetUtils.java
##########
@@ -360,6 +360,49 @@ public void testCreateSocketAddress() throws Throwable {
}
}
+ @Test
+ public void testCreateSocketAddressWithURICache() throws Throwable {
+ InetSocketAddress addr = NetUtils.createSocketAddr(
+ "127.0.0.1:12345", 1000, "myconfig", true);
+ assertEquals("127.0.0.1", addr.getAddress().getHostAddress());
+ assertEquals(12345, addr.getPort());
+
+ addr = NetUtils.createSocketAddr(
+ "127.0.0.1:12345", 1000, "myconfig", true);
+ assertEquals("127.0.0.1", addr.getAddress().getHostAddress());
+ assertEquals(12345, addr.getPort());
+
+ // ----------------------------------------------------
Review comment:
@Hexiaoqiao , thanks for your review.
To test the Cache, the same case is executed twice. There are a total of three cases. In order to divide the three cases, these comments are added.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r476180506
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/NetUtils.java
##########
@@ -218,6 +211,28 @@ public static InetSocketAddress createSocketAddr(String target,
return createSocketAddrForHost(host, port);
}
+ private static final Map<String, URI> URI_CACHE = new ConcurrentHashMap<>();
Review comment:
> Thanks @1996fanrui for your proposal here.
> A. Not sure if this improvement could impact some other modules because {{NetUtil}} is very common util for different modules rather than DFSClient only.
> B. It is better to make the scope to DFSClient only if we could based on the original description [HADOOP-17222](https://issues.apache.org/jira/browse/HADOOP-17222).
> C. add evict policy for {{URI_CACHE}}?
Another idea: add a parameter isUseURICache to the createSocketAddr() method.
isUseURICache default is false.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r478422005
##########
File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestNetUtils.java
##########
@@ -360,6 +360,49 @@ public void testCreateSocketAddress() throws Throwable {
}
}
+ @Test
+ public void testCreateSocketAddressWithURICache() throws Throwable {
+ InetSocketAddress addr = NetUtils.createSocketAddr(
+ "127.0.0.1:12345", 1000, "myconfig", true);
+ assertEquals("127.0.0.1", addr.getAddress().getHostAddress());
+ assertEquals(12345, addr.getPort());
+
+ addr = NetUtils.createSocketAddr(
+ "127.0.0.1:12345", 1000, "myconfig", true);
+ assertEquals("127.0.0.1", addr.getAddress().getHostAddress());
+ assertEquals(12345, addr.getPort());
+
+ // ----------------------------------------------------
+
+ addr = NetUtils.createSocketAddr(
+ "127.0.0.1", 1000, "myconfig", true);
+ assertEquals("127.0.0.1", addr.getAddress().getHostAddress());
+ assertEquals(1000, addr.getPort());
+
+ addr = NetUtils.createSocketAddr(
+ "127.0.0.1", 1000, "myconfig", true);
+ assertEquals("127.0.0.1", addr.getAddress().getHostAddress());
+ assertEquals(1000, addr.getPort());
+
+ // ----------------------------------------------------
+
+ try {
+ addr = NetUtils.createSocketAddr(
Review comment:
ok, I will do it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] Hexiaoqiao commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
Hexiaoqiao commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r478387611
##########
File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestNetUtils.java
##########
@@ -360,6 +360,49 @@ public void testCreateSocketAddress() throws Throwable {
}
}
+ @Test
+ public void testCreateSocketAddressWithURICache() throws Throwable {
+ InetSocketAddress addr = NetUtils.createSocketAddr(
+ "127.0.0.1:12345", 1000, "myconfig", true);
+ assertEquals("127.0.0.1", addr.getAddress().getHostAddress());
+ assertEquals(12345, addr.getPort());
+
+ addr = NetUtils.createSocketAddr(
+ "127.0.0.1:12345", 1000, "myconfig", true);
+ assertEquals("127.0.0.1", addr.getAddress().getHostAddress());
+ assertEquals(12345, addr.getPort());
+
+ // ----------------------------------------------------
Review comment:
nit: delete no meaningless annotation?
##########
File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/DfsClientConf.java
##########
@@ -211,24 +212,7 @@ public DfsClientConf(Configuration conf) {
Write.MAX_PACKETS_IN_FLIGHT_KEY,
Write.MAX_PACKETS_IN_FLIGHT_DEFAULT);
- final boolean byteArrayManagerEnabled = conf.getBoolean(
- Write.ByteArrayManager.ENABLED_KEY,
- Write.ByteArrayManager.ENABLED_DEFAULT);
- if (!byteArrayManagerEnabled) {
- writeByteArrayManagerConf = null;
- } else {
- final int countThreshold = conf.getInt(
- Write.ByteArrayManager.COUNT_THRESHOLD_KEY,
- Write.ByteArrayManager.COUNT_THRESHOLD_DEFAULT);
- final int countLimit = conf.getInt(
- Write.ByteArrayManager.COUNT_LIMIT_KEY,
- Write.ByteArrayManager.COUNT_LIMIT_DEFAULT);
- final long countResetTimePeriodMs = conf.getLong(
- Write.ByteArrayManager.COUNT_RESET_TIME_PERIOD_MS_KEY,
- Write.ByteArrayManager.COUNT_RESET_TIME_PERIOD_MS_DEFAULT);
- writeByteArrayManagerConf = new ByteArrayManager.Conf(
- countThreshold, countLimit, countResetTimePeriodMs);
- }
+ writeByteArrayManagerConf = loadWriteByteArrayManagerConf(conf);
Review comment:
nit: the changes seems no related with this ticket, Should we update it in another single one?
##########
File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestNetUtils.java
##########
@@ -360,6 +360,49 @@ public void testCreateSocketAddress() throws Throwable {
}
}
+ @Test
+ public void testCreateSocketAddressWithURICache() throws Throwable {
+ InetSocketAddress addr = NetUtils.createSocketAddr(
+ "127.0.0.1:12345", 1000, "myconfig", true);
+ assertEquals("127.0.0.1", addr.getAddress().getHostAddress());
+ assertEquals(12345, addr.getPort());
+
+ addr = NetUtils.createSocketAddr(
+ "127.0.0.1:12345", 1000, "myconfig", true);
+ assertEquals("127.0.0.1", addr.getAddress().getHostAddress());
+ assertEquals(12345, addr.getPort());
+
+ // ----------------------------------------------------
+
+ addr = NetUtils.createSocketAddr(
+ "127.0.0.1", 1000, "myconfig", true);
+ assertEquals("127.0.0.1", addr.getAddress().getHostAddress());
+ assertEquals(1000, addr.getPort());
+
+ addr = NetUtils.createSocketAddr(
+ "127.0.0.1", 1000, "myconfig", true);
+ assertEquals("127.0.0.1", addr.getAddress().getHostAddress());
+ assertEquals(1000, addr.getPort());
+
+ // ----------------------------------------------------
+
+ try {
+ addr = NetUtils.createSocketAddr(
Review comment:
nit: `addr` seems useless? no need to assign to `addr` anymore?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
liuml07 commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r478164594
##########
File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java
##########
@@ -414,6 +415,11 @@
String PREFETCH_SIZE_KEY = PREFIX + "prefetch.size";
+ String URI_CACHE_KEY = PREFIX + "uri.cache.enable";
+ boolean URI_CACHE_DEFAULT = false;
+ String URI_CACHE_EXPIRE_MS_KEY = PREFIX + "uri.cache.expire.ms";
Review comment:
Thanks, it's my bad. I could have made it clearer.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
liuml07 commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-688451138
I'm now on vacation. I will merge it this week.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-690855787
> I see `TestNameNodeRetryCacheMetrics.testRetryCacheMetrics` and ` TestMultipleNNPortQOP.testMultipleNNPortOverwriteDownStream` fail on my Linux machine with the patch. Will confirm they are not related.
>
> If no objections, I'll commit later this week. Thanks,
@liuml07 , thanks for your test.
I run these two unit tests and they are still successful. In the previous test of hadoop robot, TestMultipleNNPortQOP did not fail, only the last time it failed. But the last time I only committed an empty, my code did not change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r478399989
##########
File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/DfsClientConf.java
##########
@@ -211,24 +212,7 @@ public DfsClientConf(Configuration conf) {
Write.MAX_PACKETS_IN_FLIGHT_KEY,
Write.MAX_PACKETS_IN_FLIGHT_DEFAULT);
- final boolean byteArrayManagerEnabled = conf.getBoolean(
- Write.ByteArrayManager.ENABLED_KEY,
- Write.ByteArrayManager.ENABLED_DEFAULT);
- if (!byteArrayManagerEnabled) {
- writeByteArrayManagerConf = null;
- } else {
- final int countThreshold = conf.getInt(
- Write.ByteArrayManager.COUNT_THRESHOLD_KEY,
- Write.ByteArrayManager.COUNT_THRESHOLD_DEFAULT);
- final int countLimit = conf.getInt(
- Write.ByteArrayManager.COUNT_LIMIT_KEY,
- Write.ByteArrayManager.COUNT_LIMIT_DEFAULT);
- final long countResetTimePeriodMs = conf.getLong(
- Write.ByteArrayManager.COUNT_RESET_TIME_PERIOD_MS_KEY,
- Write.ByteArrayManager.COUNT_RESET_TIME_PERIOD_MS_DEFAULT);
- writeByteArrayManagerConf = new ByteArrayManager.Conf(
- countThreshold, countLimit, countResetTimePeriodMs);
- }
+ writeByteArrayManagerConf = loadWriteByteArrayManagerConf(conf);
Review comment:
The constructor of DfsClientConf exceeds 150 lines, and the checkstyle is not passed. So a simple refactoring was made.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r478148190
##########
File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java
##########
@@ -414,6 +415,11 @@
String PREFETCH_SIZE_KEY = PREFIX + "prefetch.size";
+ String URI_CACHE_KEY = PREFIX + "uri.cache.enable";
+ boolean URI_CACHE_DEFAULT = false;
+ String URI_CACHE_EXPIRE_MS_KEY = PREFIX + "uri.cache.expire.ms";
Review comment:
I'm so sorry, I misunderstood the meaning of jira. I thought the two parameters cache size and expire time seem a bit overkill, so the cache size was removed and the expire time was retained.
Now I will remove expire time and use 12 hours as the default value.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r476140378
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/NetUtils.java
##########
@@ -218,6 +211,28 @@ public static InetSocketAddress createSocketAddr(String target,
return createSocketAddrForHost(host, port);
}
+ private static final Map<String, URI> URI_CACHE = new ConcurrentHashMap<>();
Review comment:
Hi @Hexiaoqiao , thank you very much for your suggestions.
AB. Now Cache is just URI, URI object does not contain ip, only some fixed information. The same target corresponds to the same URI. If the URI contains some variable attributes, I think the current Cache may impact other modules, and even the hdfs client will also be affected. According to my understanding, cache URI should be safe.
C. Is `evict policy` considering that the URI will change? Or consider it may take up more memory?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 edited a comment on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
liuml07 edited a comment on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-690850909
I see `TestNameNodeRetryCacheMetrics.testRetryCacheMetrics` and ` TestMultipleNNPortQOP.testMultipleNNPortOverwriteDownStream` fail on my Linux machine with the patch. Will confirm they are not related.
If no objections, I'll commit later today. Thanks,
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r478141146
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
##########
@@ -4185,6 +4185,16 @@
</description>
</property>
+<property>
+ <name>dfs.client.read.uri.cache.enable</name>
+ <value>false</value>
+ <description>
+ If true, dfs client will use cache when creating URI based on host:port
+ to reduce the frequency of URI object creation.
+ Default is false.
Review comment:
Ok, very much agree with you
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 merged pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
liuml07 merged pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 edited a comment on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
liuml07 edited a comment on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-689153364
Was waiting for a clean QA run but could not trigger it manually. If I get no help from community, we can upload the patch file to the JIRA and from there I can trigger multiple runs to get a clean pre-commit build.
I assume failing tests are not related to this patch but it's always nice to confirm.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r478148888
##########
File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java
##########
@@ -414,6 +414,9 @@
String PREFETCH_SIZE_KEY = PREFIX + "prefetch.size";
+ String URI_CACHE_KEY = PREFIX + "uri.cache.enable";
Review comment:
ok, thanks for your review
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on a change in pull request #2241: HADOOP-17222. Create socket address combined with cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on a change in pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#discussion_r478463733
##########
File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestNetUtils.java
##########
@@ -360,6 +360,49 @@ public void testCreateSocketAddress() throws Throwable {
}
}
+ @Test
+ public void testCreateSocketAddressWithURICache() throws Throwable {
+ InetSocketAddress addr = NetUtils.createSocketAddr(
+ "127.0.0.1:12345", 1000, "myconfig", true);
+ assertEquals("127.0.0.1", addr.getAddress().getHostAddress());
+ assertEquals(12345, addr.getPort());
+
+ addr = NetUtils.createSocketAddr(
+ "127.0.0.1:12345", 1000, "myconfig", true);
+ assertEquals("127.0.0.1", addr.getAddress().getHostAddress());
+ assertEquals(12345, addr.getPort());
+
+ // ----------------------------------------------------
+
+ addr = NetUtils.createSocketAddr(
+ "127.0.0.1", 1000, "myconfig", true);
+ assertEquals("127.0.0.1", addr.getAddress().getHostAddress());
+ assertEquals(1000, addr.getPort());
+
+ addr = NetUtils.createSocketAddr(
+ "127.0.0.1", 1000, "myconfig", true);
+ assertEquals("127.0.0.1", addr.getAddress().getHostAddress());
+ assertEquals(1000, addr.getPort());
+
+ // ----------------------------------------------------
+
+ try {
+ addr = NetUtils.createSocketAddr(
Review comment:
@Hexiaoqiao I have done it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
liuml07 commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-689164399
@1996fanrui could you try the above command shared by @dineshchitlangia to trigger a new PreCommit QA run? Hopefully it will be clean next time. Thanks,
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-689272323
> Was waiting for a clean QA run but could not trigger it manually. If I get no help from community, we can upload the patch file to the JIRA and from there I can trigger multiple runs to get a clean pre-commit build.
>
> I assume failing tests are not related to this patch but it's always nice to confirm.
Hi @liuml07 , I ran all failed unit tests on my mac. Only one failed: TestHDFSContractMultipartUploader#testConcurrentUploads
It failed are not related to this patch, the community is fixing it, link: https://issues.apache.org/jira/browse/HDFS-15471
Other unit tests are successful on my mac.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
liuml07 commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-690850909
I see `TestNameNodeRetryCacheMetrics.testRetryCacheMetrics` and ` TestMultipleNNPortQOP.testMultipleNNPortOverwriteDownStream` fail on my Linux machine with the patch. Will confirm they are not related.
If no objections, I'll commit later this week. Thanks,
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] 1996fanrui commented on pull request #2241: HADOOP-17222. Create socket address combined with URI cache
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on pull request #2241:
URL: https://github.com/apache/hadoop/pull/2241#issuecomment-690855787
> I see `TestNameNodeRetryCacheMetrics.testRetryCacheMetrics` and ` TestMultipleNNPortQOP.testMultipleNNPortOverwriteDownStream` fail on my Linux machine with the patch. Will confirm they are not related.
>
> If no objections, I'll commit later this week. Thanks,
@liuml07 , thanks for your test.
I run these two unit tests and they are still successful. In the previous test of hadoop robot, TestMultipleNNPortQOP did not fail, only the last time it failed. But the last time I only committed an empty, my code did not change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org