You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Steve Vinoski <vi...@iona.com> on 2006/11/22 22:11:23 UTC
[java] flaky MultipleConnectionTest?
Anyone else noticed any flakiness with the java client
MultipleConnectionTest? It just seems to randomly pass and fail on my
machine. I tried doubling the timeout, and it still fails about half
the time.
--steve
Re: [java] flaky MultipleConnectionTest?
Posted by Steve Vinoski <vi...@iona.com>.
On Nov 23, 2006, at 9:25 AM, Robert Greig wrote:
> On 23/11/06, Steve Vinoski <vi...@iona.com> wrote:
>
>> The failure I was getting was that the listener was expecting 10
>> messages but was getting only 5 or 3 instead. What it looks like is
>> that notify is just a bit faster than notifyAll, and the way this
>> test was running, the difference added up over time. With notifyAll
>> on my machine, the test tended to take about 85 seconds on those
>> times it passed, but took only 75-78 seconds with notify, apparently
>> enough to make it pass more reliably. Are there other threads in a
>> pool somewhere? If so, that could easily explain the difference.
>
> I don't understand why it would matter that notify() is faster than
> notifyAll() in this case? i.e. semantically this test should pass with
> both? Also given that it is just sending 10 messages in-VM, it should
> be extremely quick. In any event, the timeout was 5 seconds and as the
> loop was originally crafted it waited *before* checking the count.
Well, I didn't write the test to begin with, and so don't pretend to
know all the details, but from what I can see there are multiple
receivers, each wanting to receive 10 messages. If there are enough
threads in the system such that notifyAll is needlessly notifying
threads that don't need to be notified, then the threads that
actually do need to run to increment the counts may not be getting
enough attention, and meanwhile time is passing and the 5 seconds
elapses before all receivers can collect all their messages.
> Did the messages actually get delivered? i.e was the count just wrong
> for some reason?
Other than the above, I don't know why or how the messages could be
delivered but the counts be wrong, but perhaps I'm missing something.
I would certainly not consider myself anywhere close to being an
expert at this point in time on any of this code.
>> Another thing is that with both notifyAll and notify, I also
>> sometimes get an IllegalStateException from AMQPFastProtocolHandler
>> at shutdown, line 198. I haven't looked into this yet.
>
> That is usually a symptom of another problem since it just means that
> an undecoded byte buffer has made it through to the protocol handler.
That much I know, but what I don't know yet is why it happens. Is
anyone seeing this anywhere else?
--steve
Re: [java] flaky MultipleConnectionTest?
Posted by Robert Greig <ro...@gmail.com>.
On 23/11/06, Steve Vinoski <vi...@iona.com> wrote:
> The failure I was getting was that the listener was expecting 10
> messages but was getting only 5 or 3 instead. What it looks like is
> that notify is just a bit faster than notifyAll, and the way this
> test was running, the difference added up over time. With notifyAll
> on my machine, the test tended to take about 85 seconds on those
> times it passed, but took only 75-78 seconds with notify, apparently
> enough to make it pass more reliably. Are there other threads in a
> pool somewhere? If so, that could easily explain the difference.
I don't understand why it would matter that notify() is faster than
notifyAll() in this case? i.e. semantically this test should pass with
both? Also given that it is just sending 10 messages in-VM, it should
be extremely quick. In any event, the timeout was 5 seconds and as the
loop was originally crafted it waited *before* checking the count.
Did the messages actually get delivered? i.e was the count just wrong
for some reason?
> Another thing is that with both notifyAll and notify, I also
> sometimes get an IllegalStateException from AMQPFastProtocolHandler
> at shutdown, line 198. I haven't looked into this yet.
That is usually a symptom of another problem since it just means that
an undecoded byte buffer has made it through to the protocol handler.
RG
Re: [java] flaky MultipleConnectionTest?
Posted by Steve Vinoski <vi...@iona.com>.
On Nov 22, 2006, at 6:17 PM, Robert Greig wrote:
> On 22/11/06, Steve Vinoski <vi...@iona.com> wrote:
>> On Nov 22, 2006, at 4:11 PM, Steve Vinoski wrote:
>>
>> > Anyone else noticed any flakiness with the java client
>> > MultipleConnectionTest? It just seems to randomly pass and fail on
>> > my machine. I tried doubling the timeout, and it still fails about
>> > half the time.
>>
>> Changing the notifyAll() call to notify() in this test seems to do
>> the trick. It appears that only one thread can ever be waiting at any
>> one time, so notify() should be fine.
>
> Sorry if I'm being really dim, but I don't quite understand what was
> wrong in the test? I can see that notify should be fine for this test
> but I don't see why notifyAll would cause the test to fail?
The failure I was getting was that the listener was expecting 10
messages but was getting only 5 or 3 instead. What it looks like is
that notify is just a bit faster than notifyAll, and the way this
test was running, the difference added up over time. With notifyAll
on my machine, the test tended to take about 85 seconds on those
times it passed, but took only 75-78 seconds with notify, apparently
enough to make it pass more reliably. Are there other threads in a
pool somewhere? If so, that could easily explain the difference.
Another thing is that with both notifyAll and notify, I also
sometimes get an IllegalStateException from AMQPFastProtocolHandler
at shutdown, line 198. I haven't looked into this yet.
> I ran the test on my branch and it did pass for me but I only have a
> single core CPU on my laptop. What kind of hardware are you running
> on?
Mac Powerbook, PowerPC G4 1.67 GHz, 1.5 GB RAM.
> As an aside, the comment on the test says it is slow but I think it
> could be speeded up by modifying the loop so that it didn't wait
> before checking the count, e.g.
>
> long start = System.currentTimeMillis();
> while (expected > _count)
> {
> long timeLeft = maxWait - timeSince(start);
> if (timeLeft < 0)
> {
> break;
> }
> wait(timeLeft);
> }
> return expected <= _count;
>
> I did that and the test passed in about 4 seconds rather than ~80.
Yes, I was looking at that loop as well. And with your loop, the test
consistently takes 7-8 seconds for me, an order of magnitude faster.
The fastest I saw was 5 seconds. I'll commit this change.
--steve
Re: [java] flaky MultipleConnectionTest?
Posted by Robert Greig <ro...@gmail.com>.
On 22/11/06, Steve Vinoski <vi...@iona.com> wrote:
> On Nov 22, 2006, at 4:11 PM, Steve Vinoski wrote:
>
> > Anyone else noticed any flakiness with the java client
> > MultipleConnectionTest? It just seems to randomly pass and fail on
> > my machine. I tried doubling the timeout, and it still fails about
> > half the time.
>
> Changing the notifyAll() call to notify() in this test seems to do
> the trick. It appears that only one thread can ever be waiting at any
> one time, so notify() should be fine.
Sorry if I'm being really dim, but I don't quite understand what was
wrong in the test? I can see that notify should be fine for this test
but I don't see why notifyAll would cause the test to fail?
I ran the test on my branch and it did pass for me but I only have a
single core CPU on my laptop. What kind of hardware are you running
on?
As an aside, the comment on the test says it is slow but I think it
could be speeded up by modifying the loop so that it didn't wait
before checking the count, e.g.
long start = System.currentTimeMillis();
while (expected > _count)
{
long timeLeft = maxWait - timeSince(start);
if (timeLeft < 0)
{
break;
}
wait(timeLeft);
}
return expected <= _count;
I did that and the test passed in about 4 seconds rather than ~80.
I did also wonder if _count should be volatile.
RG
Re: [java] flaky MultipleConnectionTest?
Posted by Steve Vinoski <vi...@iona.com>.
On Nov 22, 2006, at 4:11 PM, Steve Vinoski wrote:
> Anyone else noticed any flakiness with the java client
> MultipleConnectionTest? It just seems to randomly pass and fail on
> my machine. I tried doubling the timeout, and it still fails about
> half the time.
Changing the notifyAll() call to notify() in this test seems to do
the trick. It appears that only one thread can ever be waiting at any
one time, so notify() should be fine.
--steve