You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Hari Mankude (Created) (JIRA)" <ji...@apache.org> on 2012/03/23 18:07:27 UTC
[jira] [Created] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
stopproxy() is not closing the proxies correctly
------------------------------------------------
Key: HADOOP-8202
URL: https://issues.apache.org/jira/browse/HADOOP-8202
Project: Hadoop Common
Issue Type: Bug
Components: ipc
Affects Versions: 0.24.0
Reporter: Hari Mankude
Assignee: Hari Mankude
Priority: Minor
I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Suresh Srinivas (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-8202:
------------------------------------
Resolution: Fixed
Fix Version/s: 0.24.0
0.23.3
Target Version/s: 0.23.3
Status: Resolved (was: Patch Available)
I committed the patch to both trunk and 0.23.3
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 0.23.3, 0.24.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hari Mankude (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236985#comment-13236985 ]
Hari Mankude commented on HADOOP-8202:
--------------------------------------
The test was TestBackupNode. Including the relevant output from the test.
2012-03-23 12:45:31,277 INFO namenode.NameNode (NameNodeRpcServer.java:errorReport(321)) - Error report from NamenodeRegistration(localhost:64139, role=Backup Node): Shutting down.
2012-03-23 12:45:31,277 INFO namenode.FSEditLog (FSEditLog.java:releaseBackupStream(1030)) - Removing backup journal BackupJournalManager
2012-03-23 12:45:31,277 ERROR ipc.RPC (RPC.java:stopProxy(593)) - Tried to call RPC.stopProxy on an object that is not a proxy.
java.lang.IllegalArgumentException: not a proxy instance
at java.lang.reflect.Proxy.getInvocationHandler(Proxy.java:637) at org.apache.hadoop.ipc.RPC.stopProxy(RPC.java:591)
at org.apache.hadoop.hdfs.server.namenode.EditLogBackupOutputStream.abort(EditLogBackupOutputStream.java:106) at org.apache.hadoop.hdfs.server.namenode.JournalSet$JournalAndStream.abort(JournalSet.java:98)
at org.apache.hadoop.hdfs.server.namenode.JournalSet.remove(JournalSet.java:531)
at org.apache.hadoop.hdfs.server.namenode.FSEditLog.releaseBackupStream(FSEditLog.java:1031)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.releaseBackupNode(FSNamesystem.java:4663)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.errorReport(NameNodeRpcServer.java:323)
at org.apache.hadoop.hdfs.protocolPB.NamenodeProtocolServerSideTranslatorPB.errorReport(NamenodeProtocolServerSideTranslatorPB.java:125)
at org.apache.hadoop.hdfs.protocol.proto.NamenodeProtocolProtos$NamenodeProtocolService$2.callBlockingMethod(NamenodeProtocolProtos.java:8072)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:417)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:884)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1661)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1657)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1205)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1655)
2012-03-23 12:45:31,277 ERROR ipc.RPC (RPC.java:stopProxy(603)) - Could not get invocation handler null for proxy c
lass class org.apache.hadoop.hdfs.protocolPB.JournalProtocolTranslatorPB, or invocation handler is not closeable.
2012-03-23 12:45:31,278 ERROR ipc.RPC (RPC.java:stopProxy(593)) - Tried to call RPC.stopProxy on an object that is
not a proxy.java.lang.IllegalArgumentException: not a proxy instance
at java.lang.reflect.Proxy.getInvocationHandler(Proxy.java:637)
at org.apache.hadoop.ipc.RPC.stopProxy(RPC.java:591) at org.apache.hadoop.hdfs.server.namenode.BackupNode.stop(BackupNode.java:194)
at org.apache.hadoop.hdfs.server.namenode.TestBackupNode.testBackupNodeTailsEdits(TestBackupNode.java:169)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:81)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)
2012-03-23 12:45:31,278 ERROR ipc.RPC (RPC.java:stopProxy(603)) - Could not get invocation handler null for proxy class class org.apache.hadoop.hdfs.protocolPB.NamenodeProtocolTranslatorPB, or invocation handler is not closeable. ---------> Proxy was not closed.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hari Mankude (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hari Mankude updated HADOOP-8202:
---------------------------------
Status: Patch Available (was: Open)
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Suresh Srinivas (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-8202:
------------------------------------
Attachment: HADOOP-8202-4.patch
While committing I found a merge conflict - uploading the newer patch with merge conflict resolved.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hari Mankude (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236893#comment-13236893 ]
Hari Mankude commented on HADOOP-8202:
--------------------------------------
bq.
Does this fix the exception that you observed in the test, where proxy was not stopped?
Yes, it does.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hari Mankude (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hari Mankude updated HADOOP-8202:
---------------------------------
Attachment: HADOOP-8202-3.patch
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239470#comment-13239470 ]
Hudson commented on HADOOP-8202:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk #1032 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1032/])
HADOOP-8202. RPC stopProxy() does not close the proxy correctly. Contributed by Hari Mankude. (Revision 1305704)
Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1305704
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HealthMonitor.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPC.java
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 0.23.3, 0.24.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Aaron T. Myers (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238132#comment-13238132 ]
Aaron T. Myers commented on HADOOP-8202:
----------------------------------------
bq. Sure, the tests that failed were
Thanks, Hari. I agree those test failures are unrelated. TestGetBlocks is currently failing on trunk (HDFS-3143) and I've seen TestStartup and TestValidateConfigurationSettings fail spuriously before.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hari Mankude (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237383#comment-13237383 ]
Hari Mankude commented on HADOOP-8202:
--------------------------------------
testhealthmonitor failure is due to hadoop-8204.
However, I am running the entire suite of hdfs tests to make sure that nothing else is broken.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Suresh Srinivas (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238586#comment-13238586 ]
Suresh Srinivas commented on HADOOP-8202:
-----------------------------------------
bq. Instead of adding the instanceof check anywhere we use an object that might be a mock, can we instead change the protocol interfaces themselves to "extend Closeable"? That will make sure that any proxy implementations themselves take care of extending it, and also will solve the mock issue (since the mock itself will then also extend Closeable).
Given that it is done once, this patch should get committed. Todd, if you feel strongly about Closeable, please can you create a jira.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238125#comment-13238125 ]
Todd Lipcon commented on HADOOP-8202:
-------------------------------------
Instead of adding the instanceof check anywhere we use an object that might be a mock, can we instead change the protocol interfaces themselves to "extend Closeable"? That will make sure that any proxy implementations themselves take care of extending it, and also will solve the mock issue (since the mock itself will then also extend Closeable).
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Suresh Srinivas (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238588#comment-13238588 ]
Suresh Srinivas commented on HADOOP-8202:
-----------------------------------------
I am going to commit this in an hour or so. As I said in my previous comment, please create another jira, if further changes are needed.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239148#comment-13239148 ]
Hudson commented on HADOOP-8202:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #1934 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1934/])
HADOOP-8202. RPC stopProxy() does not close the proxy correctly. Contributed by Hari Mankude. (Revision 1305704)
Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1305704
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HealthMonitor.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPC.java
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 0.23.3, 0.24.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hari Mankude (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hari Mankude updated HADOOP-8202:
---------------------------------
Attachment: HADOOP-8202-1.patch
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239120#comment-13239120 ]
Todd Lipcon commented on HADOOP-8202:
-------------------------------------
This broke TestZKFailoverController, since it's now getting IllegalArgumentException trying to close the proxy.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Aaron T. Myers (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237717#comment-13237717 ]
Aaron T. Myers commented on HADOOP-8202:
----------------------------------------
Thanks for looking into the errors, Hari. We will have to do something to address the TestHealthMonitor failure before we can commit this patch - either update TestHealthMonitor so that it passes with this change as-written, or change this patch to allow for proxies/invocation handlers which do not necessarily implement Closeable.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Aaron T. Myers (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238088#comment-13238088 ]
Aaron T. Myers commented on HADOOP-8202:
----------------------------------------
Hey Hari, thanks for updating the patch to fix TestHealthMonitor.
bq. There were test failures. However, it does not seem to be related to stopproxy() changes. I will file jiras on the test failures.
Can you list here which tests failed? Can you comment on Todd's comment that he's surprised many HDFS tests didn't fail? Perhaps it's because in most tests where we mock out a protocol object, we never try to close that protocol object. Just a guess.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hari Mankude (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237667#comment-13237667 ]
Hari Mankude commented on HADOOP-8202:
--------------------------------------
hdfs tests went through. There were test failures. However, it does not seem to be related to stopproxy() changes. I will file jiras on the test failures.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241238#comment-13241238 ]
Hudson commented on HADOOP-8202:
--------------------------------
Integrated in Hadoop-Hdfs-trunk #999 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/999/])
HDFS-3156. TestDFSHAAdmin is failing post HADOOP-8202. Contributed by Aaron T. Myers. (Revision 1306517)
Result = FAILURE
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306517
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDFSHAAdmin.java
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hari Mankude (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hari Mankude updated HADOOP-8202:
---------------------------------
Attachment: HADOOP-8202-2.patch
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237785#comment-13237785 ]
Hadoop QA commented on HADOOP-8202:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12519847/HADOOP-8202-3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.fs.viewfs.TestViewFsTrash
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/774//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/774//console
This message is automatically generated.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237720#comment-13237720 ]
Todd Lipcon commented on HADOOP-8202:
-------------------------------------
I think maintaining the ability to support mockito spies/mocks for proxies is important. We use it for simulating all kinds of failure conditions -- I'm surprised you didn't have a lot of HDFS failures from the same issue.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hari Mankude (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hari Mankude updated HADOOP-8202:
---------------------------------
Attachment: HADOOP-8202.patch
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239147#comment-13239147 ]
Hudson commented on HADOOP-8202:
--------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2009 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2009/])
HADOOP-8202. RPC stopProxy() does not close the proxy correctly. Contributed by Hari Mankude. (Revision 1305704)
Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1305704
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HealthMonitor.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPC.java
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 0.23.3, 0.24.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hari Mankude (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236789#comment-13236789 ]
Hari Mankude commented on HADOOP-8202:
--------------------------------------
This jira is related to hadoop-7607
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242895#comment-13242895 ]
Todd Lipcon commented on HADOOP-8202:
-------------------------------------
This patch also introduced the following bug: if the proxy.close() function throws an IOException, then HadoopIllegalArgumentException will be thrown, claiming that the proxy doesn't implement Closeable. This is the wrong error to throw, and is a regression in behavior (failure to close due to IOE should just be a warning, as it was previously). Hari, would you mind fixing this?
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HADOOP-8202) stopproxy() is not
closing the proxies correctly
Posted by "Suresh Srinivas (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239152#comment-13239152 ]
Suresh Srinivas edited comment on HADOOP-8202 at 3/27/12 3:54 AM:
------------------------------------------------------------------
bq. This broke TestZKFailoverController, since it's now getting IllegalArgumentException trying to close the proxy.
HADOOP-8219 created.
was (Author: sureshms):
bq. This broke TestZKFailoverController, since it's now getting IllegalArgumentException trying to close the proxy.
HDFS-8219 created.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 0.23.3, 0.24.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Suresh Srinivas (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237317#comment-13237317 ]
Suresh Srinivas commented on HADOOP-8202:
-----------------------------------------
bq. HADOOP-7607
I take what you are saying - since I do not want to waste time on this.
bq. Sure, they're definitely just nits, but it's not like they're difficult to address. It almost certainly would take less time to address these comments than it does to argue about them.
In the comments, I post this kind of comments as optional - as a suggestion. If the contributor likes it/wants to address it, then it is up to him. Perhaps, adding optional/suggestion for such comments would be good, instead of cycles of nits and time wasted on addressing them. We can make better use of time.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241216#comment-13241216 ]
Hudson commented on HADOOP-8202:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk #1034 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1034/])
HDFS-3156. TestDFSHAAdmin is failing post HADOOP-8202. Contributed by Aaron T. Myers. (Revision 1306517)
Result = FAILURE
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306517
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDFSHAAdmin.java
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236949#comment-13236949 ]
Hadoop QA commented on HADOOP-8202:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12519675/HADOOP-8202.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.fs.viewfs.TestViewFsTrash
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/755//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/755//console
This message is automatically generated.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hari Mankude (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238109#comment-13238109 ]
Hari Mankude commented on HADOOP-8202:
--------------------------------------
bq.
Can you list here which tests failed? Can you comment on Todd's comment that he's surprised many HDFS tests didn't fail? Perhaps it's because in most tests where we mock out a protocol object, we never try to close that protocol object. Just a guess.
Sure, the tests that failed were
1. TestStartup
2. TestValidateConfigurationSettings
3. TestGetBlocks
I have a trivial fix for TestGetBlocks, even though unrelated to this jira. Other tests definitely don't seem to be doing stopproxy(). So, will file the jiras for now.
Regarding Todd's comment, my guess is also that we never try to close mockito based proxy objects.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239162#comment-13239162 ]
Hudson commented on HADOOP-8202:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1947 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1947/])
HADOOP-8202. RPC stopProxy() does not close the proxy correctly. Contributed by Hari Mankude. (Revision 1305704)
Result = ABORTED
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1305704
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HealthMonitor.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPC.java
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 0.23.3, 0.24.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Aaron T. Myers (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236996#comment-13236996 ]
Aaron T. Myers commented on HADOOP-8202:
----------------------------------------
bq. Not necessary - all invocation handlers in Hadoop are RpcInvocationHandlers and they implement Closeable.
Can you guarantee that all future invocation handlers in Hadoop will implement Closeable?
bq. I think this seems wholly unnecessary. Currently stopProxy is completely failing. I know that multiple times things have changed in this part of the code, and has been broken for some time.
This is a great reason to add a test - so that such things don't regress in the future.
bq. This is your coding style. I am not sure if it should be followed by every one.
Sure, it's a preference, but that in itself isn't a good reason to not address the comment. Can you comment on why it's better to split the reasons for an error across several log statements?
bq. I want to look at the ProtocolTranslator and why it is needed in detail. In this part of the code clearly ProtocolTranslator is not required.
If you have a better way to implement RPC.getServerAddress, I'm happy to hear it.
bq. The test was TestBackupNode.
Thanks for the info, Hari.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Aaron T. Myers (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237278#comment-13237278 ]
Aaron T. Myers commented on HADOOP-8202:
----------------------------------------
bq. The problem with the code that was written is, it silently ignored the error and just printed a log and did not indicate the error. This is what is being fixed now.
Note that when HADOOP-7607 was being implemented, it was a conscious decision to log a warning instead of throwing, so as to maintain backward compatibility with the previous implementation. See [this comment|https://issues.apache.org/jira/browse/HADOOP-7607?focusedCommentId=13097236&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13097236]. We're now making a conscious decision to change that here, which is fine, but it should be noted.
bq. HADOOP-7607 did not add tests either. Hence this bug. I suggest, lets practice what we preach!
HADOOP-7607 didn't introduce this bug. HADOOP-7607 was just a refactor, hence why it had no tests.
This bug was introduced because of the following events:
# It used to be that all proxy objects used by client classes were directly referenced by RPC engines. During this time, the code in RPC#stopProxy worked just fine, but required users of proxy objects that wrapped other proxy objects to hold a reference to the underlying proxy object just for the purpose of calling RPC#stopProxy. This is why for a long time DFSClient had two references to ClientProtocol objects - one to call methods on, the other to call RPC#stopProxy on.
# HADOOP-7607 refactored the code in RPC#stopProxy to actually call close on the invocation handler of the proxy object directly, instead of going through the RPCEngine. This would allow the invocation handler to bubble this down to underlying proxies. This was committed in September of 2011.
# We began to introduce protocol translators for protobuf support in December of 2011. This caused the code in RPC#stopProxy to stop actually releasing underlying resources when RPC#stopProxy was called with a translator object provided as the argument, since the objects we were calling RPC#stopProxy on were no longer actual proxy objects, and hence Proxy#getInvocationHandler would fail. Unfortunately, no one noticed this until now. The introduction of protocol translators also caused RPC#getServerAddress to break, but again no one noticed. I introduced the ProtocolTranslator interface because I needed RPC#getServerAddress to work in order to implement HDFS-2792.
bq. On a related note, I am also not happy for simple changes, we keep mandating adding unit tests. Some times, it is okay to use judgement call and not add unnecessary tests.
Sure, I agree with you. The question here is what constitutes a "simple change."
FWIW, we often add tests for "simple changes," not to prove that the new code works, but to prevent it from ever regressing in the future. See HDFS-3099 as an example.
bq. I am observing that our code reviews are becoming too strict. Not every patch I review should look like the code I would write. As long it is correct, follows coding standards, it should be good. I have been seeing some comments these days, to say, can we call variable name as ioe instead of e. I believe, we should relax these.
Sure, they're definitely just nits, but it's not like they're difficult to address. It almost certainly would take less time to address these comments than it does to argue about them.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Suresh Srinivas (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236888#comment-13236888 ]
Suresh Srinivas commented on HADOOP-8202:
-----------------------------------------
Does this fix the exception that you observed in the test, where proxy was not stopped?
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Suresh Srinivas (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239152#comment-13239152 ]
Suresh Srinivas commented on HADOOP-8202:
-----------------------------------------
bq. This broke TestZKFailoverController, since it's now getting IllegalArgumentException trying to close the proxy.
HDFS-8219 created.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 0.23.3, 0.24.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240594#comment-13240594 ]
Hudson commented on HADOOP-8202:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #1939 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1939/])
HDFS-3156. TestDFSHAAdmin is failing post HADOOP-8202. Contributed by Aaron T. Myers. (Revision 1306517)
Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306517
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDFSHAAdmin.java
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hari Mankude (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242922#comment-13242922 ]
Hari Mankude commented on HADOOP-8202:
--------------------------------------
Will take a look.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Aaron T. Myers (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236931#comment-13236931 ]
Aaron T. Myers commented on HADOOP-8202:
----------------------------------------
Also, given that implementing the ProtocolTranslator interface is necessary in order for {{RPC#getServerAddress}} to work, perhaps a better solution would be to leave {{RPC#stopProxy}} as it is now, and change the protocol translators that currently implement {{Closeable}} to instead implement {{ProtocolTrnaslator}}.
Thoughts?
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Suresh Srinivas (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237207#comment-13237207 ]
Suresh Srinivas commented on HADOOP-8202:
-----------------------------------------
The problem with the code that was written is, it silently ignored the error and just printed a log and did not indicate the error. This is what is being fixed now.
bq. Can you guarantee that all future invocation handlers in Hadoop will implement Closeable?
Write the code with either proxy is closeable or has an invocation handler that is closeable. If that is not the case, then it is programming error! Throw RuntimeException, so it is found early and not silently ignored.
bq. This is a great reason to add a test - so that such things don't regress in the future.
HADOOP-7607 did not add tests either. Hence this bug. I suggest, lets practice what we preach!
On a related note, I am also not happy for simple changes, we keep mandating adding unit tests. Some times, it is okay to use judgement call and not add unnecessary tests. Adding useless tests comes at a cost. When we did the federation feature, we spent half the time fixing poorly documented and lame tests. Not that these tests were finding bugs, the tests simply did not work well the code changes.
That said, Hari, if you want you can add tests.
bq. Sure, it's a preference, but that in itself isn't a good reason to not address the comment. Can you comment on why it's better to split the reasons for an error across several log statements?
You cannot argue matter of taste. Is the code incorrect? Else, I like this code better, because I do not have to handle exception twice, but in one single place around closeable. I am observing that our code reviews are becoming too strict. Not every patch I review should look like the code I would write. As long it is correct, follows coding standards, it should be good. I have been seeing some comments these days, to say, can we call variable name as ioe instead of e. I believe, we should relax these.
bq. If you have a better way to implement RPC.getServerAddress, I'm happy to hear it.
Perhaps this is the only way. I need to look into it. But that is not required while stopping proxy. It is orthogonal.
Hari, after thinking a bit, I believe we should throw HadoopIllegalArgumentException, if either the proxy is not closeable or does not have Invocation handler.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238597#comment-13238597 ]
Todd Lipcon commented on HADOOP-8202:
-------------------------------------
Sure, go ahead and commit. Thanks.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239423#comment-13239423 ]
Hudson commented on HADOOP-8202:
--------------------------------
Integrated in Hadoop-Hdfs-trunk #997 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/997/])
HADOOP-8202. RPC stopProxy() does not close the proxy correctly. Contributed by Hari Mankude. (Revision 1305704)
Result = FAILURE
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1305704
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HealthMonitor.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPC.java
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 0.23.3, 0.24.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hari Mankude (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237271#comment-13237271 ]
Hari Mankude commented on HADOOP-8202:
--------------------------------------
Including a patch that takes care of concerns
1. Throws HadoopIllegalArgumentexception if proxy is not closeable
2. Throws HadoopIllegalArgumentexception if invocation handler is not closeable
3. New test to catch the error scenario when a null proxy is used.
4. Modified existing negative test so that it does not fail since exception is thrown.
5. Fixed javadoc comments
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hari Mankude (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237706#comment-13237706 ]
Hari Mankude commented on HADOOP-8202:
--------------------------------------
hadoop-common tests also passed except for testheatlhmonitor. Looking at the testhealthmonitor test, it is creating a proxy using mockito and I am guessing that this is not closeable. This is the reason for the error.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238713#comment-13238713 ]
Hadoop QA commented on HADOOP-8202:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12519992/HADOOP-8202-4.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/783//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/783//console
This message is automatically generated.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242343#comment-13242343 ]
Hudson commented on HADOOP-8202:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk #1035 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1035/])
HDFS-3156. TestDFSHAAdmin is failing post HADOOP-8202. Contributed by Aaron T. Myers. (Revision 1306517)
Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306517
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDFSHAAdmin.java
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236863#comment-13236863 ]
Hadoop QA commented on HADOOP-8202:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12519670/HADOOP-8202.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.fs.viewfs.TestViewFsTrash
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/754//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/754//artifact/trunk/hadoop-common-project/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/754//console
This message is automatically generated.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Aaron T. Myers (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237295#comment-13237295 ]
Aaron T. Myers commented on HADOOP-8202:
----------------------------------------
The latest patch looks good to me. +1 pending Jenkins.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240623#comment-13240623 ]
Hudson commented on HADOOP-8202:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1952 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1952/])
HDFS-3156. TestDFSHAAdmin is failing post HADOOP-8202. Contributed by Aaron T. Myers. (Revision 1306517)
Result = ABORTED
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306517
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDFSHAAdmin.java
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237331#comment-13237331 ]
Hadoop QA commented on HADOOP-8202:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12519747/HADOOP-8202-2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.ha.TestHealthMonitor
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/772//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/772//console
This message is automatically generated.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hari Mankude (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hari Mankude updated HADOOP-8202:
---------------------------------
Attachment: HADOOP-8202.patch
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Aaron T. Myers (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236908#comment-13236908 ]
Aaron T. Myers commented on HADOOP-8202:
----------------------------------------
Hi Hari, a few comments:
# "c = (Closeable) Proxy.getInvocationHandler(proxy);" - this could potentially cause an uncaught ClassCastException, if the InvocationHandler itself doesn't implement Closeable.
# Given the above, the error message at the bottom of the message should perhaps also include "or invocation handler does not implement Closeable"
# 'LOG.error(... + "or does not provide invocation handler for proxy class"' - should put a space after "class"
# 'LOG.error("Cannot close proxy since it is null ");' - unnecessary whitespace at the end of the string.
# Seems like it shouldn't be too tough to write a test for this with some mock objects.
# Rather than have a single catch-all error message at the bottom, and return early to avoid it, I think it'd be better to only ever log a single error, and include the relevant information which caused the failure to close the proxy in that log message.
Also:
bq. Does this fix the exception that you observed in the test, where proxy was not stopped?
What exception? In what test?
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Suresh Srinivas (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236948#comment-13236948 ]
Suresh Srinivas commented on HADOOP-8202:
-----------------------------------------
bq. "c = (Closeable) Proxy.getInvocationHandler(proxy);" - this could potentially cause an uncaught ClassCastException, if the InvocationHandler itself doesn't implement Closeable.
Not necessary - all invocation handlers in Hadoop are RpcInvocationHandlers and they implement Closeable.
bq. Seems like it shouldn't be too tough to write a test for this with some mock objects.
I think this seems wholly unnecessary. Currently stopProxy is completely failing. I know that multiple times things have changed in this part of the code, and has been broken for some time.
I am +1 without any tests.
bq. Rather than have a single catch-all error message at the bottom, and return early to avoid it, I think it'd be better to only ever log a single error, and include the relevant information which caused the failure to close the proxy in that log message.
This is your coding style. I am not sure if it should be followed by every one.
Again, I am okay to commit this, once we have log statement from the existing test.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Suresh Srinivas (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236952#comment-13236952 ]
Suresh Srinivas commented on HADOOP-8202:
-----------------------------------------
bq. Also, given that implementing the ProtocolTranslator interface is necessary in order for RPC#getServerAddress to work, perhaps a better solution would be to leave RPC#stopProxy as it is now, and change the protocol translators that currently implement Closeable to instead implement ProtocolTrnaslator.
I want to look at the ProtocolTranslator and why it is needed in detail. In this part of the code clearly ProtocolTranslator is not required.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hari Mankude (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237787#comment-13237787 ]
Hari Mankude commented on HADOOP-8202:
--------------------------------------
testviewfstrash failure is unrelated to this change.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240590#comment-13240590 ]
Hudson commented on HADOOP-8202:
--------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2014 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2014/])
HDFS-3156. TestDFSHAAdmin is failing post HADOOP-8202. Contributed by Aaron T. Myers. (Revision 1306517)
Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306517
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDFSHAAdmin.java
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13237005#comment-13237005 ]
Hadoop QA commented on HADOOP-8202:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12519700/HADOOP-8202-1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/758//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/758//console
This message is automatically generated.
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Attachments: HADOOP-8202-1.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8202) stopproxy() is not closing the
proxies correctly
Posted by "Suresh Srinivas (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-8202:
------------------------------------
Hadoop Flags: Reviewed
> stopproxy() is not closing the proxies correctly
> ------------------------------------------------
>
> Key: HADOOP-8202
> URL: https://issues.apache.org/jira/browse/HADOOP-8202
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.24.0
> Reporter: Hari Mankude
> Assignee: Hari Mankude
> Priority: Minor
> Fix For: 0.23.3, 0.24.0
>
> Attachments: HADOOP-8202-1.patch, HADOOP-8202-2.patch, HADOOP-8202-3.patch, HADOOP-8202-4.patch, HADOOP-8202.patch, HADOOP-8202.patch
>
>
> I was running testbackupnode and noticed that NNprotocol proxy was not being closed. Talked with Suresh and he observed that most of the protocols do not implement ProtocolTranslator and hence the logic in stopproxy() does not work. Instead, since all of them are closeable, Suresh suggested that closeable property should be used at close.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira