You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Hadoop QA (JIRA)" <ji...@apache.org> on 2018/10/11 12:46:00 UTC
[jira] [Commented] (HDDS-555) RandomKeyGenerator runs not closing
the XceiverClient properly
[ https://issues.apache.org/jira/browse/HDDS-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16646381#comment-16646381 ]
Hadoop QA commented on HDDS-555:
--------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 43s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 3 new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m 22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m 45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m 57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 15m 16s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-ozone/integration-test {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 20s{color} | {color:red} integration-test in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 10m 6s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 10m 6s{color} | {color:red} root in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 3m 22s{color} | {color:orange} root: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 31s{color} | {color:red} integration-test in the patch failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 43s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-ozone/integration-test {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s{color} | {color:green} client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 29s{color} | {color:green} client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 28s{color} | {color:red} integration-test in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 32s{color} | {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 94m 25s{color} | {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 |
| JIRA Issue | HDDS-555 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12943388/HDDS-555.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 5c72cfc20b40 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 8a37983 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| mvninstall | https://builds.apache.org/job/PreCommit-HDDS-Build/1358/artifact/out/patch-mvninstall-hadoop-ozone_integration-test.txt |
| compile | https://builds.apache.org/job/PreCommit-HDDS-Build/1358/artifact/out/patch-compile-root.txt |
| javac | https://builds.apache.org/job/PreCommit-HDDS-Build/1358/artifact/out/patch-compile-root.txt |
| checkstyle | https://builds.apache.org/job/PreCommit-HDDS-Build/1358/artifact/out/diff-checkstyle-root.txt |
| mvnsite | https://builds.apache.org/job/PreCommit-HDDS-Build/1358/artifact/out/patch-mvnsite-hadoop-ozone_integration-test.txt |
| unit | https://builds.apache.org/job/PreCommit-HDDS-Build/1358/artifact/out/patch-unit-hadoop-ozone_integration-test.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HDDS-Build/1358/testReport/ |
| Max. process+thread count | 459 (vs. ulimit of 10000) |
| modules | C: hadoop-hdds/client hadoop-ozone/client hadoop-ozone/integration-test U: . |
| Console output | https://builds.apache.org/job/PreCommit-HDDS-Build/1358/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> RandomKeyGenerator runs not closing the XceiverClient properly
> --------------------------------------------------------------
>
> Key: HDDS-555
> URL: https://issues.apache.org/jira/browse/HDDS-555
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Shashikant Banerjee
> Assignee: Mukul Kumar Singh
> Priority: Major
> Attachments: HDDS-555.001.patch
>
>
> The issue is with closing of the xceiverClients in Ozone. The below shows the leak in XceiverClientGrpc. Similar issue can be seen intermittentkly with XceiverClientRatis as well as shown in https://issues.apache.org/jira/browse/RATIS-318 as well.
> {code:java}
> Sep 26, 2018 8:11:01 PM org.apache.ratis.shaded.io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference cleanQueue
> SEVERE: *~*~*~ Channel ManagedChannelImpl{logId=136, target=192.168.1.2:50712} was not shutdown properly!!! ~*~*~*
> Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true.
> java.lang.RuntimeException: ManagedChannel allocation site
> at org.apache.ratis.shaded.io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference.<init>(ManagedChannelOrphanWrapper.java:103)
> at org.apache.ratis.shaded.io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:53)
> at org.apache.ratis.shaded.io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:44)
> at org.apache.ratis.shaded.io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:410)
> at org.apache.hadoop.hdds.scm.XceiverClientGrpc.connect(XceiverClientGrpc.java:92)
> at org.apache.hadoop.hdds.scm.XceiverClientManager$2.call(XceiverClientManager.java:159)
> at org.apache.hadoop.hdds.scm.XceiverClientManager$2.call(XceiverClientManager.java:144)
> at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4767)
> at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
> at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
> at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
> at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4764)
> at org.apache.hadoop.hdds.scm.XceiverClientManager.getClient(XceiverClientManager.java:143)
> at org.apache.hadoop.hdds.scm.XceiverClientManager.acquireClient(XceiverClientManager.java:122)
> at org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.checkKeyLocationInfo(ChunkGroupOutputStream.java:192)
> at org.apache.hadoop.ozone.client.io.ChunkGroupOutputStream.addPreallocateBlocks(ChunkGroupOutputStream.java:180)
> at org.apache.hadoop.ozone.client.rpc.RpcClient.createKey(RpcClient.java:472)
> at org.apache.hadoop.ozone.client.OzoneBucket.createKey(OzoneBucket.java:262)
> at org.apache.hadoop.ozone.freon.RandomKeyGenerator$OfflineProcessor.run(RandomKeyGenerator.java:601)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org