You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by Patrick Hunt <ph...@apache.org> on 2008/07/30 18:32:52 UTC
ZooKeeper test harness needs work
I'm soliciting ideas on how to improve the reliability and usability
(creating new tests in particular) of our current test harness.
Hudson and some users are seeing intermittent failures, primarily due to
timing related issue in test setup; the test starts a server, runs some
tests, then shuts down the server, loop to the next test. There is some
code in ClientBase that's supposed to provide a latch for the server
startup, but we also have a number of "sleeps" in the test setup,
without which the tests fail more frequently (so something is still
busted). In particular I want to make it easier to write server tests
and to remove the need for sleeps as this causes the unit tests to run
slowly.
Additionally we are seeing a need to tests clients in addition to the
server (much of our current testing is related to verifying the correct
function of the zk server). In this case we are not currently able to
test any client failure handling cases (such as disconnect handling) as
we are running against a fully functional zk server.
I'm thinking we should do two things:
1) create a better test harness for the server
2) implement a mock ZooKeeper.java that has similar semantics as zk
server proper but has the capability to inject/reproduce/verify various
error scenarios for client testing.
If you have any ideas/suggestions/comments/etc.. or would like to work
on/with please let me know.
Patrick
Re: ZooKeeper test harness needs work
Posted by Hiram Chirino <hi...@hiramchirino.com>.
I was dual core Intel Mac Book Pro.
On Fri, Aug 1, 2008 at 4:57 PM, Patrick Hunt <ph...@apache.org> wrote:
> Could be, but I don't think I'm seeing that in the test failure cases I've
> seen lately. In some cases the VM itself was crashing! Notice ZOOKEEPER-2
> was seeing NPEs in the QuorumPeer.
>
> I did notice in AsyncTest that it's closing the server(s) before closing the
> clients, which is causing alot of noise in the logs (planning to patch
> that).
>
> What OS/cpu type configuration are you on? I'm on Ubuntu on 1core cpu.
>
> Patrick
>
> Hiram Chirino wrote:
>>
>> Lots of times when a test runs it seems like the ports are already
>> bound. Anybody else see this? Is this the cause of the intermittent
>> failures you see?
>>
>> Regards,
>> Hiram
>>
>> On Wed, Jul 30, 2008 at 12:32 PM, Patrick Hunt <ph...@apache.org> wrote:
>>>
>>> I'm soliciting ideas on how to improve the reliability and usability
>>> (creating new tests in particular) of our current test harness.
>>>
>>> Hudson and some users are seeing intermittent failures, primarily due to
>>> timing related issue in test setup; the test starts a server, runs some
>>> tests, then shuts down the server, loop to the next test. There is some
>>> code
>>> in ClientBase that's supposed to provide a latch for the server startup,
>>> but
>>> we also have a number of "sleeps" in the test setup, without which the
>>> tests
>>> fail more frequently (so something is still busted). In particular I want
>>> to
>>> make it easier to write server tests and to remove the need for sleeps as
>>> this causes the unit tests to run slowly.
>>>
>>> Additionally we are seeing a need to tests clients in addition to the
>>> server
>>> (much of our current testing is related to verifying the correct function
>>> of
>>> the zk server). In this case we are not currently able to test any client
>>> failure handling cases (such as disconnect handling) as we are running
>>> against a fully functional zk server.
>>>
>>> I'm thinking we should do two things:
>>>
>>> 1) create a better test harness for the server
>>> 2) implement a mock ZooKeeper.java that has similar semantics as zk
>>> server
>>> proper but has the capability to inject/reproduce/verify various error
>>> scenarios for client testing.
>>>
>>> If you have any ideas/suggestions/comments/etc.. or would like to work
>>> on/with please let me know.
>>>
>>> Patrick
>>>
>>
>>
>>
>
--
Regards,
Hiram
Blog: http://hiramchirino.com
Open Source SOA
http://open.iona.com
Re: ZooKeeper test harness needs work
Posted by Patrick Hunt <ph...@apache.org>.
Could be, but I don't think I'm seeing that in the test failure cases
I've seen lately. In some cases the VM itself was crashing! Notice
ZOOKEEPER-2 was seeing NPEs in the QuorumPeer.
I did notice in AsyncTest that it's closing the server(s) before closing
the clients, which is causing alot of noise in the logs (planning to
patch that).
What OS/cpu type configuration are you on? I'm on Ubuntu on 1core cpu.
Patrick
Hiram Chirino wrote:
> Lots of times when a test runs it seems like the ports are already
> bound. Anybody else see this? Is this the cause of the intermittent
> failures you see?
>
> Regards,
> Hiram
>
> On Wed, Jul 30, 2008 at 12:32 PM, Patrick Hunt <ph...@apache.org> wrote:
>> I'm soliciting ideas on how to improve the reliability and usability
>> (creating new tests in particular) of our current test harness.
>>
>> Hudson and some users are seeing intermittent failures, primarily due to
>> timing related issue in test setup; the test starts a server, runs some
>> tests, then shuts down the server, loop to the next test. There is some code
>> in ClientBase that's supposed to provide a latch for the server startup, but
>> we also have a number of "sleeps" in the test setup, without which the tests
>> fail more frequently (so something is still busted). In particular I want to
>> make it easier to write server tests and to remove the need for sleeps as
>> this causes the unit tests to run slowly.
>>
>> Additionally we are seeing a need to tests clients in addition to the server
>> (much of our current testing is related to verifying the correct function of
>> the zk server). In this case we are not currently able to test any client
>> failure handling cases (such as disconnect handling) as we are running
>> against a fully functional zk server.
>>
>> I'm thinking we should do two things:
>>
>> 1) create a better test harness for the server
>> 2) implement a mock ZooKeeper.java that has similar semantics as zk server
>> proper but has the capability to inject/reproduce/verify various error
>> scenarios for client testing.
>>
>> If you have any ideas/suggestions/comments/etc.. or would like to work
>> on/with please let me know.
>>
>> Patrick
>>
>
>
>
Re: ZooKeeper test harness needs work
Posted by Hiram Chirino <hi...@hiramchirino.com>.
Lots of times when a test runs it seems like the ports are already
bound. Anybody else see this? Is this the cause of the intermittent
failures you see?
Regards,
Hiram
On Wed, Jul 30, 2008 at 12:32 PM, Patrick Hunt <ph...@apache.org> wrote:
> I'm soliciting ideas on how to improve the reliability and usability
> (creating new tests in particular) of our current test harness.
>
> Hudson and some users are seeing intermittent failures, primarily due to
> timing related issue in test setup; the test starts a server, runs some
> tests, then shuts down the server, loop to the next test. There is some code
> in ClientBase that's supposed to provide a latch for the server startup, but
> we also have a number of "sleeps" in the test setup, without which the tests
> fail more frequently (so something is still busted). In particular I want to
> make it easier to write server tests and to remove the need for sleeps as
> this causes the unit tests to run slowly.
>
> Additionally we are seeing a need to tests clients in addition to the server
> (much of our current testing is related to verifying the correct function of
> the zk server). In this case we are not currently able to test any client
> failure handling cases (such as disconnect handling) as we are running
> against a fully functional zk server.
>
> I'm thinking we should do two things:
>
> 1) create a better test harness for the server
> 2) implement a mock ZooKeeper.java that has similar semantics as zk server
> proper but has the capability to inject/reproduce/verify various error
> scenarios for client testing.
>
> If you have any ideas/suggestions/comments/etc.. or would like to work
> on/with please let me know.
>
> Patrick
>
--
Regards,
Hiram
Blog: http://hiramchirino.com
Open Source SOA
http://open.iona.com