You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "Patrick Hunt (JIRA)" <ji...@apache.org> on 2009/07/15 18:47:15 UTC
[jira] Created: (ZOOKEEPER-473) cleanup junit tests to eliminate
false positives due to "socket reuse" and failure to close client
cleanup junit tests to eliminate false positives due to "socket reuse" and failure to close client
--------------------------------------------------------------------------------------------------
Key: ZOOKEEPER-473
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-473
Project: Zookeeper
Issue Type: Improvement
Components: tests
Affects Versions: 3.2.0
Reporter: Patrick Hunt
Assignee: Patrick Hunt
Fix For: 3.3.0
The junit tests will occasionally fail on hudson due to a few problems:
1) socket in use - sometimes the code/testcode will fail to close a socket, either that or tcp wait
kicks in and the kernel holds onto the socket, either way subsequent tests that use this same
port will fail
2) sometimes the tests fail to close clients, this results in clients attempting to re-connect to
the server started as part of a subsequent test (the timeout is very long). this causes a few
problems, typically extra load on the server or more likely unusual changes to the server that the
next test(s) don't expect. also causes problems when trying to debug the test as the old clients
spew messages into the log.
we see this on hudson, occasionally failures. We also saw this when working on ZOOKEEPER-397, the testng
port. testng does not support "fork mode" for running the tests. as a result all the tests (all) run in a single
vm instance, start to finish. so this problem is exacerbated significantly in that case. this patch only
fixes the problem in junit btw, it does not move to testng
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-473) cleanup junit tests to eliminate
false positives due to "socket reuse" and failure to close client
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12732167#action_12732167 ]
Patrick Hunt commented on ZOOKEEPER-473:
----------------------------------------
btw -- @Test is fine with 1.5 - we had that in most places already, this patch just adds to a few tests that were missing.
> cleanup junit tests to eliminate false positives due to "socket reuse" and failure to close client
> --------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-473
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-473
> Project: Zookeeper
> Issue Type: Improvement
> Components: tests
> Affects Versions: 3.2.0
> Reporter: Patrick Hunt
> Assignee: Patrick Hunt
> Fix For: 3.3.0
>
> Attachments: ZOOKEEPER-473.patch, ZOOKEEPER-473.patch
>
>
> The junit tests will occasionally fail on hudson due to a few problems:
> 1) socket in use - sometimes the code/testcode will fail to close a socket, either that or tcp wait
> kicks in and the kernel holds onto the socket, either way subsequent tests that use this same
> port will fail
> 2) sometimes the tests fail to close clients, this results in clients attempting to re-connect to
> the server started as part of a subsequent test (the timeout is very long). this causes a few
> problems, typically extra load on the server or more likely unusual changes to the server that the
> next test(s) don't expect. also causes problems when trying to debug the test as the old clients
> spew messages into the log.
> we see this on hudson, occasionally failures. We also saw this when working on ZOOKEEPER-397, the testng
> port. testng does not support "fork mode" for running the tests. as a result all the tests (all) run in a single
> vm instance, start to finish. so this problem is exacerbated significantly in that case. this patch only
> fixes the problem in junit btw, it does not move to testng
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-473) cleanup junit tests to eliminate
false positives due to "socket reuse" and failure to close client
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated ZOOKEEPER-473:
------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. thanks pat.
> cleanup junit tests to eliminate false positives due to "socket reuse" and failure to close client
> --------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-473
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-473
> Project: Zookeeper
> Issue Type: Improvement
> Components: tests
> Affects Versions: 3.2.0
> Reporter: Patrick Hunt
> Assignee: Patrick Hunt
> Fix For: 3.3.0
>
> Attachments: ZOOKEEPER-473.patch, ZOOKEEPER-473.patch
>
>
> The junit tests will occasionally fail on hudson due to a few problems:
> 1) socket in use - sometimes the code/testcode will fail to close a socket, either that or tcp wait
> kicks in and the kernel holds onto the socket, either way subsequent tests that use this same
> port will fail
> 2) sometimes the tests fail to close clients, this results in clients attempting to re-connect to
> the server started as part of a subsequent test (the timeout is very long). this causes a few
> problems, typically extra load on the server or more likely unusual changes to the server that the
> next test(s) don't expect. also causes problems when trying to debug the test as the old clients
> spew messages into the log.
> we see this on hudson, occasionally failures. We also saw this when working on ZOOKEEPER-397, the testng
> port. testng does not support "fork mode" for running the tests. as a result all the tests (all) run in a single
> vm instance, start to finish. so this problem is exacerbated significantly in that case. this patch only
> fixes the problem in junit btw, it does not move to testng
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-473) cleanup junit tests to eliminate
false positives due to "socket reuse" and failure to close client
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-473:
-----------------------------------
Status: Patch Available (was: Open)
> cleanup junit tests to eliminate false positives due to "socket reuse" and failure to close client
> --------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-473
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-473
> Project: Zookeeper
> Issue Type: Improvement
> Components: tests
> Affects Versions: 3.2.0
> Reporter: Patrick Hunt
> Assignee: Patrick Hunt
> Fix For: 3.3.0
>
> Attachments: ZOOKEEPER-473.patch, ZOOKEEPER-473.patch
>
>
> The junit tests will occasionally fail on hudson due to a few problems:
> 1) socket in use - sometimes the code/testcode will fail to close a socket, either that or tcp wait
> kicks in and the kernel holds onto the socket, either way subsequent tests that use this same
> port will fail
> 2) sometimes the tests fail to close clients, this results in clients attempting to re-connect to
> the server started as part of a subsequent test (the timeout is very long). this causes a few
> problems, typically extra load on the server or more likely unusual changes to the server that the
> next test(s) don't expect. also causes problems when trying to debug the test as the old clients
> spew messages into the log.
> we see this on hudson, occasionally failures. We also saw this when working on ZOOKEEPER-397, the testng
> port. testng does not support "fork mode" for running the tests. as a result all the tests (all) run in a single
> vm instance, start to finish. so this problem is exacerbated significantly in that case. this patch only
> fixes the problem in junit btw, it does not move to testng
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-473) cleanup junit tests to eliminate
false positives due to "socket reuse" and failure to close client
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-473:
-----------------------------------
Attachment: ZOOKEEPER-473.patch
latest patch moves back to console for test log4j output
> cleanup junit tests to eliminate false positives due to "socket reuse" and failure to close client
> --------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-473
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-473
> Project: Zookeeper
> Issue Type: Improvement
> Components: tests
> Affects Versions: 3.2.0
> Reporter: Patrick Hunt
> Assignee: Patrick Hunt
> Fix For: 3.3.0
>
> Attachments: ZOOKEEPER-473.patch, ZOOKEEPER-473.patch
>
>
> The junit tests will occasionally fail on hudson due to a few problems:
> 1) socket in use - sometimes the code/testcode will fail to close a socket, either that or tcp wait
> kicks in and the kernel holds onto the socket, either way subsequent tests that use this same
> port will fail
> 2) sometimes the tests fail to close clients, this results in clients attempting to re-connect to
> the server started as part of a subsequent test (the timeout is very long). this causes a few
> problems, typically extra load on the server or more likely unusual changes to the server that the
> next test(s) don't expect. also causes problems when trying to debug the test as the old clients
> spew messages into the log.
> we see this on hudson, occasionally failures. We also saw this when working on ZOOKEEPER-397, the testng
> port. testng does not support "fork mode" for running the tests. as a result all the tests (all) run in a single
> vm instance, start to finish. so this problem is exacerbated significantly in that case. this patch only
> fixes the problem in junit btw, it does not move to testng
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-473) cleanup junit tests to eliminate
false positives due to "socket reuse" and failure to close client
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731720#action_12731720 ]
Mahadev konar commented on ZOOKEEPER-473:
-----------------------------------------
the patch looks good... this should finally get the tests running on my mac (it used to fail because of the bind exceptions, mac isnt that good at releasing ports). I had 2 comments -
- I think we should still keep the systemoutput/log in the same file and let hudson have the output as a huge file, the only reason being that it allows the system otuputs to match with the logging using log4j. I would be useful to see all the logging and sys output/ error in a single file for a given test. what do you think?
- also, another check does @Test tags work with javac 1.5? I am not sure if we still want to support 1.5.
> cleanup junit tests to eliminate false positives due to "socket reuse" and failure to close client
> --------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-473
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-473
> Project: Zookeeper
> Issue Type: Improvement
> Components: tests
> Affects Versions: 3.2.0
> Reporter: Patrick Hunt
> Assignee: Patrick Hunt
> Fix For: 3.3.0
>
> Attachments: ZOOKEEPER-473.patch
>
>
> The junit tests will occasionally fail on hudson due to a few problems:
> 1) socket in use - sometimes the code/testcode will fail to close a socket, either that or tcp wait
> kicks in and the kernel holds onto the socket, either way subsequent tests that use this same
> port will fail
> 2) sometimes the tests fail to close clients, this results in clients attempting to re-connect to
> the server started as part of a subsequent test (the timeout is very long). this causes a few
> problems, typically extra load on the server or more likely unusual changes to the server that the
> next test(s) don't expect. also causes problems when trying to debug the test as the old clients
> spew messages into the log.
> we see this on hudson, occasionally failures. We also saw this when working on ZOOKEEPER-397, the testng
> port. testng does not support "fork mode" for running the tests. as a result all the tests (all) run in a single
> vm instance, start to finish. so this problem is exacerbated significantly in that case. this patch only
> fixes the problem in junit btw, it does not move to testng
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-473) cleanup junit tests to eliminate
false positives due to "socket reuse" and failure to close client
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-473:
-----------------------------------
Status: Open (was: Patch Available)
> cleanup junit tests to eliminate false positives due to "socket reuse" and failure to close client
> --------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-473
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-473
> Project: Zookeeper
> Issue Type: Improvement
> Components: tests
> Affects Versions: 3.2.0
> Reporter: Patrick Hunt
> Assignee: Patrick Hunt
> Fix For: 3.3.0
>
> Attachments: ZOOKEEPER-473.patch, ZOOKEEPER-473.patch
>
>
> The junit tests will occasionally fail on hudson due to a few problems:
> 1) socket in use - sometimes the code/testcode will fail to close a socket, either that or tcp wait
> kicks in and the kernel holds onto the socket, either way subsequent tests that use this same
> port will fail
> 2) sometimes the tests fail to close clients, this results in clients attempting to re-connect to
> the server started as part of a subsequent test (the timeout is very long). this causes a few
> problems, typically extra load on the server or more likely unusual changes to the server that the
> next test(s) don't expect. also causes problems when trying to debug the test as the old clients
> spew messages into the log.
> we see this on hudson, occasionally failures. We also saw this when working on ZOOKEEPER-397, the testng
> port. testng does not support "fork mode" for running the tests. as a result all the tests (all) run in a single
> vm instance, start to finish. so this problem is exacerbated significantly in that case. this patch only
> fixes the problem in junit btw, it does not move to testng
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-473) cleanup junit tests to eliminate
false positives due to "socket reuse" and failure to close client
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-473:
-----------------------------------
Status: Patch Available (was: Open)
> cleanup junit tests to eliminate false positives due to "socket reuse" and failure to close client
> --------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-473
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-473
> Project: Zookeeper
> Issue Type: Improvement
> Components: tests
> Affects Versions: 3.2.0
> Reporter: Patrick Hunt
> Assignee: Patrick Hunt
> Fix For: 3.3.0
>
> Attachments: ZOOKEEPER-473.patch
>
>
> The junit tests will occasionally fail on hudson due to a few problems:
> 1) socket in use - sometimes the code/testcode will fail to close a socket, either that or tcp wait
> kicks in and the kernel holds onto the socket, either way subsequent tests that use this same
> port will fail
> 2) sometimes the tests fail to close clients, this results in clients attempting to re-connect to
> the server started as part of a subsequent test (the timeout is very long). this causes a few
> problems, typically extra load on the server or more likely unusual changes to the server that the
> next test(s) don't expect. also causes problems when trying to debug the test as the old clients
> spew messages into the log.
> we see this on hudson, occasionally failures. We also saw this when working on ZOOKEEPER-397, the testng
> port. testng does not support "fork mode" for running the tests. as a result all the tests (all) run in a single
> vm instance, start to finish. so this problem is exacerbated significantly in that case. this patch only
> fixes the problem in junit btw, it does not move to testng
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-473) cleanup junit tests to eliminate
false positives due to "socket reuse" and failure to close client
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-473:
-----------------------------------
Attachment: ZOOKEEPER-473.patch
this patch addresses the stated problems.
Additional note -- I've moved all of the log4j output to a logfile in the build/test/logs directory.
this was necessary for testng since it considers "stdout" special - and we were logging all
of the log4j to the console previously. I think this change is an improvement because:
1) hudson console information will be much easier to view (currently huge)
2) there's a single file to open/view for all log output. in general this is convienient
and specifically it's nice if you are trying to see if a test at time t is causing problems
with a subsequent t+n test (like for this jira)
3) the junit test output log files for each test now are much more readable/parsable, they
contain just the results of the test (incl excep) and any stdout/stderr.
Please review and commit asap.
> cleanup junit tests to eliminate false positives due to "socket reuse" and failure to close client
> --------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-473
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-473
> Project: Zookeeper
> Issue Type: Improvement
> Components: tests
> Affects Versions: 3.2.0
> Reporter: Patrick Hunt
> Assignee: Patrick Hunt
> Fix For: 3.3.0
>
> Attachments: ZOOKEEPER-473.patch
>
>
> The junit tests will occasionally fail on hudson due to a few problems:
> 1) socket in use - sometimes the code/testcode will fail to close a socket, either that or tcp wait
> kicks in and the kernel holds onto the socket, either way subsequent tests that use this same
> port will fail
> 2) sometimes the tests fail to close clients, this results in clients attempting to re-connect to
> the server started as part of a subsequent test (the timeout is very long). this causes a few
> problems, typically extra load on the server or more likely unusual changes to the server that the
> next test(s) don't expect. also causes problems when trying to debug the test as the old clients
> spew messages into the log.
> we see this on hudson, occasionally failures. We also saw this when working on ZOOKEEPER-397, the testng
> port. testng does not support "fork mode" for running the tests. as a result all the tests (all) run in a single
> vm instance, start to finish. so this problem is exacerbated significantly in that case. this patch only
> fixes the problem in junit btw, it does not move to testng
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-473) cleanup junit tests to eliminate
false positives due to "socket reuse" and failure to close client
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12735502#action_12735502 ]
Hudson commented on ZOOKEEPER-473:
----------------------------------
Integrated in ZooKeeper-trunk #393 (See [http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/393/])
> cleanup junit tests to eliminate false positives due to "socket reuse" and failure to close client
> --------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-473
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-473
> Project: Zookeeper
> Issue Type: Improvement
> Components: tests
> Affects Versions: 3.2.0
> Reporter: Patrick Hunt
> Assignee: Patrick Hunt
> Fix For: 3.3.0
>
> Attachments: ZOOKEEPER-473.patch, ZOOKEEPER-473.patch
>
>
> The junit tests will occasionally fail on hudson due to a few problems:
> 1) socket in use - sometimes the code/testcode will fail to close a socket, either that or tcp wait
> kicks in and the kernel holds onto the socket, either way subsequent tests that use this same
> port will fail
> 2) sometimes the tests fail to close clients, this results in clients attempting to re-connect to
> the server started as part of a subsequent test (the timeout is very long). this causes a few
> problems, typically extra load on the server or more likely unusual changes to the server that the
> next test(s) don't expect. also causes problems when trying to debug the test as the old clients
> spew messages into the log.
> we see this on hudson, occasionally failures. We also saw this when working on ZOOKEEPER-397, the testng
> port. testng does not support "fork mode" for running the tests. as a result all the tests (all) run in a single
> vm instance, start to finish. so this problem is exacerbated significantly in that case. this patch only
> fixes the problem in junit btw, it does not move to testng
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.