You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Kihwal Lee (JIRA)" <ji...@apache.org> on 2011/08/02 01:10:27 UTC
[jira] [Created] (HADOOP-7492) RPC client should deal with the IP
address changes
RPC client should deal with the IP address changes
--------------------------------------------------
Key: HADOOP-7492
URL: https://issues.apache.org/jira/browse/HADOOP-7492
Project: Hadoop Common
Issue Type: Improvement
Components: ipc
Affects Versions: 0.23.0
Reporter: Kihwal Lee
Priority: Blocker
Fix For: 0.23.0
Make sure HADOOP-7472 is forward ported to 0.23.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7492) Add automated test for the RPC IP
addr change detection and reconnect feature
Posted by "Kihwal Lee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee updated HADOOP-7492:
-------------------------------
Issue Type: Test (was: Improvement)
> Add automated test for the RPC IP addr change detection and reconnect feature
> -----------------------------------------------------------------------------
>
> Key: HADOOP-7492
> URL: https://issues.apache.org/jira/browse/HADOOP-7492
> Project: Hadoop Common
> Issue Type: Test
> Components: ipc
> Affects Versions: 0.23.0
> Reporter: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: TestRPCAddressChange.patch
>
>
> Make sure HADOOP-7472 is forward ported to 0.23 by adding an automated test.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7492) Add automated test for the RPC IP
addr change detection and reconnect feature
Posted by "Thomas Graves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Graves updated HADOOP-7492:
----------------------------------
Priority: Major (was: Blocker)
> Add automated test for the RPC IP addr change detection and reconnect feature
> -----------------------------------------------------------------------------
>
> Key: HADOOP-7492
> URL: https://issues.apache.org/jira/browse/HADOOP-7492
> Project: Hadoop Common
> Issue Type: Test
> Components: ipc
> Affects Versions: 0.23.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Fix For: 0.24.0
>
> Attachments: TestRPCAddressChange-1.patch, TestRPCAddressChange.patch
>
>
> Make sure HADOOP-7472 is forward ported to 0.23 by adding an automated test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7492) Add automated test for the RPC IP
addr change detection and reconnect feature
Posted by "Kihwal Lee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13083473#comment-13083473 ]
Kihwal Lee commented on HADOOP-7492:
------------------------------------
In the attached patch, the constructor of InetSocketAddress is mocked to return wrong or correct address. It requires PowerMock to run. I will make it "patch available" once HADOOP-7537 is resolved.
> Add automated test for the RPC IP addr change detection and reconnect feature
> -----------------------------------------------------------------------------
>
> Key: HADOOP-7492
> URL: https://issues.apache.org/jira/browse/HADOOP-7492
> Project: Hadoop Common
> Issue Type: Test
> Components: ipc
> Affects Versions: 0.23.0
> Reporter: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: TestRPCAddressChange.patch
>
>
> Make sure HADOOP-7472 is forward ported to 0.23 by adding an automated test.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7492) Add automated test for the RPC IP
addr change detection and reconnect feature
Posted by "Kihwal Lee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee updated HADOOP-7492:
-------------------------------
Description:
Make sure HADOOP-7472 is forward ported to 0.23 by adding an automated test.
was:Make sure HADOOP-7472 is forward ported to 0.23.
Summary: Add automated test for the RPC IP addr change detection and reconnect feature (was: RPC client should deal with the IP address changes)
> Add automated test for the RPC IP addr change detection and reconnect feature
> -----------------------------------------------------------------------------
>
> Key: HADOOP-7492
> URL: https://issues.apache.org/jira/browse/HADOOP-7492
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.23.0
> Reporter: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0
>
>
> Make sure HADOOP-7472 is forward ported to 0.23 by adding an automated test.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7492) Add automated test for the RPC IP
addr change detection and reconnect feature
Posted by "Kihwal Lee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13083368#comment-13083368 ]
Kihwal Lee commented on HADOOP-7492:
------------------------------------
HADOOP-7472 has been committed to TRUNK. The testing of IP address change detection was done manually, but we will have a way to automate the test with HADOOP-7537.
> Add automated test for the RPC IP addr change detection and reconnect feature
> -----------------------------------------------------------------------------
>
> Key: HADOOP-7492
> URL: https://issues.apache.org/jira/browse/HADOOP-7492
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.23.0
> Reporter: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0
>
>
> Make sure HADOOP-7472 is forward ported to 0.23 by adding an automated test.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7492) Add automated test for the RPC IP
addr change detection and reconnect feature
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated HADOOP-7492:
----------------------------
Fix Version/s: (was: 0.24.0)
> Add automated test for the RPC IP addr change detection and reconnect feature
> -----------------------------------------------------------------------------
>
> Key: HADOOP-7492
> URL: https://issues.apache.org/jira/browse/HADOOP-7492
> Project: Hadoop Common
> Issue Type: Test
> Components: ipc
> Affects Versions: 0.23.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Attachments: TestRPCAddressChange-1.patch, TestRPCAddressChange.patch
>
>
> Make sure HADOOP-7472 is forward ported to 0.23 by adding an automated test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-7492) Add automated test for the RPC IP
addr change detection and reconnect feature
Posted by "Kihwal Lee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee reassigned HADOOP-7492:
----------------------------------
Assignee: Kihwal Lee
> Add automated test for the RPC IP addr change detection and reconnect feature
> -----------------------------------------------------------------------------
>
> Key: HADOOP-7492
> URL: https://issues.apache.org/jira/browse/HADOOP-7492
> Project: Hadoop Common
> Issue Type: Test
> Components: ipc
> Affects Versions: 0.23.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: TestRPCAddressChange.patch
>
>
> Make sure HADOOP-7472 is forward ported to 0.23 by adding an automated test.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7492) Add automated test for the RPC IP
addr change detection and reconnect feature
Posted by "Kihwal Lee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089686#comment-13089686 ]
Kihwal Lee commented on HADOOP-7492:
------------------------------------
Given the state of HADOOP-7537, this test might not make it. I will wait until Daryn's changes and then think about other ways of testing it.
> Add automated test for the RPC IP addr change detection and reconnect feature
> -----------------------------------------------------------------------------
>
> Key: HADOOP-7492
> URL: https://issues.apache.org/jira/browse/HADOOP-7492
> Project: Hadoop Common
> Issue Type: Test
> Components: ipc
> Affects Versions: 0.23.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: TestRPCAddressChange-1.patch, TestRPCAddressChange.patch
>
>
> Make sure HADOOP-7472 is forward ported to 0.23 by adding an automated test.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7492) Add automated test for the RPC IP
addr change detection and reconnect feature
Posted by "Kihwal Lee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee updated HADOOP-7492:
-------------------------------
Attachment: TestRPCAddressChange.patch
{{Test output - requires PowerMockito}}
{noformat}
2011-08-11 14:03:03,281 INFO ipc.Server (Server.java:run(349)) - Starting Socket Reader #1 for port 49429
2011-08-11 14:03:03,311 INFO ipc.Server (Server.java:run(642)) - IPC Server Responder: starting
2011-08-11 14:03:03,314 INFO ipc.Server (Server.java:run(473)) - IPC Server listener on 49429: starting
2011-08-11 14:03:03,315 INFO ipc.Server (Server.java:run(1466)) - IPC Server handler 0 on 49429: starting
2011-08-11 14:03:03,318 INFO ipc.Server (Server.java:run(349)) - Starting Socket Reader #1 for port 34748
2011-08-11 14:03:03,319 INFO ipc.Server (Server.java:run(642)) - IPC Server Responder: starting
2011-08-11 14:03:03,319 INFO ipc.Server (Server.java:run(473)) - IPC Server listener on 34748: starting
2011-08-11 14:03:03,331 INFO ipc.Server (Server.java:run(1466)) - IPC Server handler 0 on 34748: starting
2011-08-11 14:03:05,301 INFO ipc.Server (Server.java:stop(1715)) - Stopping server on 49429
2011-08-11 14:03:05,302 INFO ipc.Server (Server.java:run(1539)) - IPC Server handler 0 on 49429: exiting
2011-08-11 14:03:05,302 INFO ipc.Server (Server.java:run(505)) - Stopping IPC Server listener on 49429
2011-08-11 14:03:05,303 INFO ipc.Server (Server.java:run(647)) - Stopping IPC Server Responder
2011-08-11 14:03:06,382 INFO ipc.Client (Client.java:handleConnectionFailure(670)) - Retrying connect to server: localhost.localdomain/127.0.0.1:49429. Already tried 0 time(s).
2011-08-11 14:03:07,400 INFO ipc.Client (Client.java:handleConnectionFailure(670)) - Retrying connect to server: localhost.localdomain/127.0.0.1:49429. Already tried 1 time(s).
2011-08-11 14:03:07,413 WARN ipc.Client (Client.java:updateAddress(422)) - Address change detected. Old: localhost.localdomain/127.0.0.1:49429 New: /127.0.0.1:34748
2011-08-11 14:03:08,426 INFO ipc.Client (Client.java:handleConnectionFailure(670)) - Retrying connect to server: /127.0.0.1:34748. Already tried 0 time(s).
2011-08-11 14:03:08,490 INFO ipc.Server (Server.java:stop(1715)) - Stopping server on 34748
2011-08-11 14:03:08,490 INFO ipc.Server (Server.java:run(1539)) - IPC Server handler 0 on 34748: exiting
2011-08-11 14:03:08,490 INFO ipc.Server (Server.java:run(505)) - Stopping IPC Server listener on 34748
2011-08-11 14:03:08,491 INFO ipc.Server (Server.java:run(647)) - Stopping IPC Server Responder
{noformat}
> Add automated test for the RPC IP addr change detection and reconnect feature
> -----------------------------------------------------------------------------
>
> Key: HADOOP-7492
> URL: https://issues.apache.org/jira/browse/HADOOP-7492
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.23.0
> Reporter: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: TestRPCAddressChange.patch
>
>
> Make sure HADOOP-7472 is forward ported to 0.23 by adding an automated test.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7492) Add automated test for the RPC IP
addr change detection and reconnect feature
Posted by "Suresh Srinivas (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13193161#comment-13193161 ]
Suresh Srinivas commented on HADOOP-7492:
-----------------------------------------
Kihwals, why is this marked a blocker?
> Add automated test for the RPC IP addr change detection and reconnect feature
> -----------------------------------------------------------------------------
>
> Key: HADOOP-7492
> URL: https://issues.apache.org/jira/browse/HADOOP-7492
> Project: Hadoop Common
> Issue Type: Test
> Components: ipc
> Affects Versions: 0.23.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.24.0
>
> Attachments: TestRPCAddressChange-1.patch, TestRPCAddressChange.patch
>
>
> Make sure HADOOP-7472 is forward ported to 0.23 by adding an automated test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7492) Add automated test for the RPC IP
addr change detection and reconnect feature
Posted by "Kihwal Lee (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13193486#comment-13193486 ]
Kihwal Lee commented on HADOOP-7492:
------------------------------------
A while back one of the reviewers asked to file this and mark it as a blocker. I guess the situation has changed.
> Add automated test for the RPC IP addr change detection and reconnect feature
> -----------------------------------------------------------------------------
>
> Key: HADOOP-7492
> URL: https://issues.apache.org/jira/browse/HADOOP-7492
> Project: Hadoop Common
> Issue Type: Test
> Components: ipc
> Affects Versions: 0.23.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.24.0
>
> Attachments: TestRPCAddressChange-1.patch, TestRPCAddressChange.patch
>
>
> Make sure HADOOP-7472 is forward ported to 0.23 by adding an automated test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7492) Add automated test for the RPC IP
addr change detection and reconnect feature
Posted by "Kihwal Lee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee updated HADOOP-7492:
-------------------------------
Attachment: TestRPCAddressChange-1.patch
The new patch incorporates Suresh's review comments.
> Add automated test for the RPC IP addr change detection and reconnect feature
> -----------------------------------------------------------------------------
>
> Key: HADOOP-7492
> URL: https://issues.apache.org/jira/browse/HADOOP-7492
> Project: Hadoop Common
> Issue Type: Test
> Components: ipc
> Affects Versions: 0.23.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: TestRPCAddressChange-1.patch, TestRPCAddressChange.patch
>
>
> Make sure HADOOP-7472 is forward ported to 0.23 by adding an automated test.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7492) Add automated test for the RPC IP
addr change detection and reconnect feature
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088041#comment-13088041 ]
Suresh Srinivas commented on HADOOP-7492:
-----------------------------------------
Comments:
# Can you please add in testAddrChange() method comments what the behavior at the client that is being tested, instead of terse, "address change detection by the RPC client". Description that covers, RPC client on seeing connection failure, re-resolves the address and connects to the new address would be good. This will make the test easy for others to understand.
# Can you please set in conf, "ipc.client.connect.max.retries" to 10 (even though default is 10, which might change) to ensure more than two attempts are made for the test to pass. done in two retries? Also along with whenNew could you add a comment that return old address for two retry attempts and resolve to a new address in the third attempt?
# Nit: LocaTestServer#call() has indentation issues
# Nit: Please add to class javadoc "Unit tests for RPC addr change detection in {@link Client}
# Nit: typo attemp
> Add automated test for the RPC IP addr change detection and reconnect feature
> -----------------------------------------------------------------------------
>
> Key: HADOOP-7492
> URL: https://issues.apache.org/jira/browse/HADOOP-7492
> Project: Hadoop Common
> Issue Type: Test
> Components: ipc
> Affects Versions: 0.23.0
> Reporter: Kihwal Lee
> Assignee: Kihwal Lee
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: TestRPCAddressChange.patch
>
>
> Make sure HADOOP-7472 is forward ported to 0.23 by adding an automated test.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira