[jira] [Updated] (ZOOKEEPER-2183) Change test port assignments to improve uniqueness of ports for multiple concurrent test processes on the same host.

["Chris Nauroth (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ZOOKEEPER-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Nauroth updated ZOOKEEPER-2183:
-------------------------------------
    Attachment: ZOOKEEPER-2183.001.patch

I'm attaching a patch.  Here is a summary of the changes:
# build.xml: Add support for concurrent execution of tests.  The default is still a single JUnit process (same as today), but you can override it by passing a command line argument.  This would require everyone to upgrade to Ant 1.9.4.  I don't know what version Jenkins runs, so if it fails on this, then I'll provide another patch without the build.xml change.  On my fairly wimpy CentOS VM, I can do a full test run with {{ant -Dtest.junit.threads=8 -Dtest.output=yes clean test-core-java}} in ~10-15 minutes, down from ~60 minutes.
# {{PortAssignment}}: I rewrote this to assign ports by binding to an ephemeral port temporarily.  I documented that there is still a very small race condition risk, but it's going to be much better than the current situation where multiple JUnit processes starting from the same base port number could collide.  I did explore trying to bootstrap test servers with port 0 to eliminate this race condition completely, but that would have required intrusive changes in product code and tests that expect to know all their port numbers upfront before running anything.
# Multiple test suites are updated to skip starting {{JettyAdminServer}}, so that we don't get a {{BindException}} when they all try to use the same port.  All test suites that inherit from {{ZKTestCase}} already do this.
# Multiple tests are updated to remove "+ 1000" logic from their port assignments.  This looks like it was an attempt to offset port numbers a bit in certain tests to try to increase the probability of uniqueness.  This is no longer necessary with the new {{PortAssignment}}.  Also, the logic is incorrect, because it could inadvertently cause a port collision or overflow past the valid range for port numbers.

I've tried multiple concurrent test runs with this patch, and there are no signs of a {{BindException}}, so I think this covers the changes we need in port assignments.

> Change test port assignments to improve uniqueness of ports for multiple concurrent test processes on the same host.
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: ZOOKEEPER-2183
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2183
>             Project: ZooKeeper
>          Issue Type: Improvement
>          Components: tests
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: ZOOKEEPER-2183.001.patch
>
>
> Tests use {{PortAssignment#unique}} for assignment of the ports to bind during tests.  Currently, this method works by using a monotonically increasing counter from a static starting point.  Generally, this is sufficient to achieve uniqueness within a single JVM process, but it does not achieve uniqueness across multiple processes on the same host.  This can cause tests to get bind errors if there are multiple pre-commit jobs running concurrently on the same Jenkins host.  This also prevents running tests in parallel to improve the speed of pre-commit runs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)