You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by Patrick Hunt <ph...@apache.org> on 2011/10/28 01:40:15 UTC

ZOOKEEPER-962 -- FollowerResyncConcurrencyTest failing frequently

In my CI environment I have some lower powered (virt) hardware,
therein I see this test failing frequently:

https://issues.apache.org/jira/browse/ZOOKEEPER-962
junit.framework.AssertionFailedError: Should have same number of
ephemerals in both followers expected:<11241> but was:<14001>
	at org.apache.zookeeper.test.FollowerResyncConcurrencyTest.verifyState(FollowerResyncConcurrencyTest.java:381)
	at org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes(FollowerResyncConcurrencyTest.java:186)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52)

In this test I see all kinds of Thread.sleep calls which makes me
suspect that it might be the tests failing due to slow h/w. However
while trunk is failing branch 3.3 has not seen this issue. I'm not
running testing on branch 3.4 (I'm starting now).

I don't see this on apache ZK trunk testing.


Thoughts? Could this be false positives from the test, or something
more serious?

Patrick

Re: ZOOKEEPER-962 -- FollowerResyncConcurrencyTest failing frequently

Posted by Mahadev Konar <ma...@hortonworks.com>.
Thanks Pat. Just reviewed.

mahadev

On Thu, Oct 27, 2011 at 11:00 PM, Patrick Hunt <ph...@apache.org> wrote:
> I posted some patches to https://issues.apache.org/jira/browse/ZOOKEEPER-1264
>
> Fixes the problem for me (problem was in the test afaict).
>
> If someone reviews the patches I'll commit them.
>
> Patrick
>
> On Thu, Oct 27, 2011 at 9:24 PM, Patrick Hunt <ph...@apache.org> wrote:
>> Looks like the 1 second session timeout may be at fault. I'm delving
>> into it and will open a jira.
>>
>> Patrick
>>
>> On Thu, Oct 27, 2011 at 7:40 PM, Camille Fournier <ca...@apache.org> wrote:
>>> That is not good... Don't suppose you could debug it a bit, or send logs at
>>> least.
>>>
>>> C
>>>
>>> From my phone
>>> On Oct 27, 2011 8:01 PM, "Patrick Hunt" <ph...@apache.org> wrote:
>>>
>>>> My first run on 3.4 branch resulted in:
>>>>
>>>> junit.framework.AssertionFailedError: Should have same number of
>>>> ephemerals in both followers expected:<11741> but was:<14001>
>>>>        at
>>>> org.apache.zookeeper.test.FollowerResyncConcurrencyTest.verifyState(FollowerResyncConcurrencyTest.java:400)
>>>>        at
>>>> org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes(FollowerResyncConcurrencyTest.java:196)
>>>>        at
>>>> org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52)
>>>>
>>>> Patrick
>>>>
>>>> On Thu, Oct 27, 2011 at 4:40 PM, Patrick Hunt <ph...@apache.org> wrote:
>>>> > In my CI environment I have some lower powered (virt) hardware,
>>>> > therein I see this test failing frequently:
>>>> >
>>>> > https://issues.apache.org/jira/browse/ZOOKEEPER-962
>>>> > junit.framework.AssertionFailedError: Should have same number of
>>>> > ephemerals in both followers expected:<11241> but was:<14001>
>>>> >        at
>>>> org.apache.zookeeper.test.FollowerResyncConcurrencyTest.verifyState(FollowerResyncConcurrencyTest.java:381)
>>>> >        at
>>>> org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes(FollowerResyncConcurrencyTest.java:186)
>>>> >        at
>>>> org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52)
>>>> >
>>>> > In this test I see all kinds of Thread.sleep calls which makes me
>>>> > suspect that it might be the tests failing due to slow h/w. However
>>>> > while trunk is failing branch 3.3 has not seen this issue. I'm not
>>>> > running testing on branch 3.4 (I'm starting now).
>>>> >
>>>> > I don't see this on apache ZK trunk testing.
>>>> >
>>>> >
>>>> > Thoughts? Could this be false positives from the test, or something
>>>> > more serious?
>>>> >
>>>> > Patrick
>>>> >
>>>>
>>>
>>
>

Re: ZOOKEEPER-962 -- FollowerResyncConcurrencyTest failing frequently

Posted by Patrick Hunt <ph...@apache.org>.
I posted some patches to https://issues.apache.org/jira/browse/ZOOKEEPER-1264

Fixes the problem for me (problem was in the test afaict).

If someone reviews the patches I'll commit them.

Patrick

On Thu, Oct 27, 2011 at 9:24 PM, Patrick Hunt <ph...@apache.org> wrote:
> Looks like the 1 second session timeout may be at fault. I'm delving
> into it and will open a jira.
>
> Patrick
>
> On Thu, Oct 27, 2011 at 7:40 PM, Camille Fournier <ca...@apache.org> wrote:
>> That is not good... Don't suppose you could debug it a bit, or send logs at
>> least.
>>
>> C
>>
>> From my phone
>> On Oct 27, 2011 8:01 PM, "Patrick Hunt" <ph...@apache.org> wrote:
>>
>>> My first run on 3.4 branch resulted in:
>>>
>>> junit.framework.AssertionFailedError: Should have same number of
>>> ephemerals in both followers expected:<11741> but was:<14001>
>>>        at
>>> org.apache.zookeeper.test.FollowerResyncConcurrencyTest.verifyState(FollowerResyncConcurrencyTest.java:400)
>>>        at
>>> org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes(FollowerResyncConcurrencyTest.java:196)
>>>        at
>>> org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52)
>>>
>>> Patrick
>>>
>>> On Thu, Oct 27, 2011 at 4:40 PM, Patrick Hunt <ph...@apache.org> wrote:
>>> > In my CI environment I have some lower powered (virt) hardware,
>>> > therein I see this test failing frequently:
>>> >
>>> > https://issues.apache.org/jira/browse/ZOOKEEPER-962
>>> > junit.framework.AssertionFailedError: Should have same number of
>>> > ephemerals in both followers expected:<11241> but was:<14001>
>>> >        at
>>> org.apache.zookeeper.test.FollowerResyncConcurrencyTest.verifyState(FollowerResyncConcurrencyTest.java:381)
>>> >        at
>>> org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes(FollowerResyncConcurrencyTest.java:186)
>>> >        at
>>> org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52)
>>> >
>>> > In this test I see all kinds of Thread.sleep calls which makes me
>>> > suspect that it might be the tests failing due to slow h/w. However
>>> > while trunk is failing branch 3.3 has not seen this issue. I'm not
>>> > running testing on branch 3.4 (I'm starting now).
>>> >
>>> > I don't see this on apache ZK trunk testing.
>>> >
>>> >
>>> > Thoughts? Could this be false positives from the test, or something
>>> > more serious?
>>> >
>>> > Patrick
>>> >
>>>
>>
>

Re: ZOOKEEPER-962 -- FollowerResyncConcurrencyTest failing frequently

Posted by Patrick Hunt <ph...@apache.org>.
Looks like the 1 second session timeout may be at fault. I'm delving
into it and will open a jira.

Patrick

On Thu, Oct 27, 2011 at 7:40 PM, Camille Fournier <ca...@apache.org> wrote:
> That is not good... Don't suppose you could debug it a bit, or send logs at
> least.
>
> C
>
> From my phone
> On Oct 27, 2011 8:01 PM, "Patrick Hunt" <ph...@apache.org> wrote:
>
>> My first run on 3.4 branch resulted in:
>>
>> junit.framework.AssertionFailedError: Should have same number of
>> ephemerals in both followers expected:<11741> but was:<14001>
>>        at
>> org.apache.zookeeper.test.FollowerResyncConcurrencyTest.verifyState(FollowerResyncConcurrencyTest.java:400)
>>        at
>> org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes(FollowerResyncConcurrencyTest.java:196)
>>        at
>> org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52)
>>
>> Patrick
>>
>> On Thu, Oct 27, 2011 at 4:40 PM, Patrick Hunt <ph...@apache.org> wrote:
>> > In my CI environment I have some lower powered (virt) hardware,
>> > therein I see this test failing frequently:
>> >
>> > https://issues.apache.org/jira/browse/ZOOKEEPER-962
>> > junit.framework.AssertionFailedError: Should have same number of
>> > ephemerals in both followers expected:<11241> but was:<14001>
>> >        at
>> org.apache.zookeeper.test.FollowerResyncConcurrencyTest.verifyState(FollowerResyncConcurrencyTest.java:381)
>> >        at
>> org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes(FollowerResyncConcurrencyTest.java:186)
>> >        at
>> org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52)
>> >
>> > In this test I see all kinds of Thread.sleep calls which makes me
>> > suspect that it might be the tests failing due to slow h/w. However
>> > while trunk is failing branch 3.3 has not seen this issue. I'm not
>> > running testing on branch 3.4 (I'm starting now).
>> >
>> > I don't see this on apache ZK trunk testing.
>> >
>> >
>> > Thoughts? Could this be false positives from the test, or something
>> > more serious?
>> >
>> > Patrick
>> >
>>
>

Re: ZOOKEEPER-962 -- FollowerResyncConcurrencyTest failing frequently

Posted by Camille Fournier <ca...@apache.org>.
That is not good... Don't suppose you could debug it a bit, or send logs at
least.

C

>From my phone
On Oct 27, 2011 8:01 PM, "Patrick Hunt" <ph...@apache.org> wrote:

> My first run on 3.4 branch resulted in:
>
> junit.framework.AssertionFailedError: Should have same number of
> ephemerals in both followers expected:<11741> but was:<14001>
>        at
> org.apache.zookeeper.test.FollowerResyncConcurrencyTest.verifyState(FollowerResyncConcurrencyTest.java:400)
>        at
> org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes(FollowerResyncConcurrencyTest.java:196)
>        at
> org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52)
>
> Patrick
>
> On Thu, Oct 27, 2011 at 4:40 PM, Patrick Hunt <ph...@apache.org> wrote:
> > In my CI environment I have some lower powered (virt) hardware,
> > therein I see this test failing frequently:
> >
> > https://issues.apache.org/jira/browse/ZOOKEEPER-962
> > junit.framework.AssertionFailedError: Should have same number of
> > ephemerals in both followers expected:<11241> but was:<14001>
> >        at
> org.apache.zookeeper.test.FollowerResyncConcurrencyTest.verifyState(FollowerResyncConcurrencyTest.java:381)
> >        at
> org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes(FollowerResyncConcurrencyTest.java:186)
> >        at
> org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52)
> >
> > In this test I see all kinds of Thread.sleep calls which makes me
> > suspect that it might be the tests failing due to slow h/w. However
> > while trunk is failing branch 3.3 has not seen this issue. I'm not
> > running testing on branch 3.4 (I'm starting now).
> >
> > I don't see this on apache ZK trunk testing.
> >
> >
> > Thoughts? Could this be false positives from the test, or something
> > more serious?
> >
> > Patrick
> >
>

Re: ZOOKEEPER-962 -- FollowerResyncConcurrencyTest failing frequently

Posted by Patrick Hunt <ph...@apache.org>.
My first run on 3.4 branch resulted in:

junit.framework.AssertionFailedError: Should have same number of
ephemerals in both followers expected:<11741> but was:<14001>
	at org.apache.zookeeper.test.FollowerResyncConcurrencyTest.verifyState(FollowerResyncConcurrencyTest.java:400)
	at org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes(FollowerResyncConcurrencyTest.java:196)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52)

Patrick

On Thu, Oct 27, 2011 at 4:40 PM, Patrick Hunt <ph...@apache.org> wrote:
> In my CI environment I have some lower powered (virt) hardware,
> therein I see this test failing frequently:
>
> https://issues.apache.org/jira/browse/ZOOKEEPER-962
> junit.framework.AssertionFailedError: Should have same number of
> ephemerals in both followers expected:<11241> but was:<14001>
>        at org.apache.zookeeper.test.FollowerResyncConcurrencyTest.verifyState(FollowerResyncConcurrencyTest.java:381)
>        at org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes(FollowerResyncConcurrencyTest.java:186)
>        at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:52)
>
> In this test I see all kinds of Thread.sleep calls which makes me
> suspect that it might be the tests failing due to slow h/w. However
> while trunk is failing branch 3.3 has not seen this issue. I'm not
> running testing on branch 3.4 (I'm starting now).
>
> I don't see this on apache ZK trunk testing.
>
>
> Thoughts? Could this be false positives from the test, or something
> more serious?
>
> Patrick
>