You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Rajith Attapattu <ra...@gmail.com> on 2009/10/23 03:26:36 UTC
cpp.testprofile failures
I have temporarily excluded the following tests from the cpp test
profiles to get them green.
All these tests have been added within the last 2 weeks.
I will investigate them one by one to ensure if any of the failures
are genuine or needs to be permanently disabled.
Are these tests run against the 0-10 Java Broker?
If so I'd be interested to know the outcome.
org.apache.qpid.test.unit.publish.DirtyTrasactedPubilshTest
org.apache.qpid.test.unit.ack.FailoverBeforeConsumingRecoverTest
org.apache.qpid.test.unit.ack.AcknowledgeAfterFailoverTest
org.apache.qpid.test.unit.ack.AcknowledgeAfterFailoverOnMessageTest
org.apache.qpid.test.client.RollbackOrderTest#testOrderingAfterRollbackOnMessage
Regards,
Rajith Attapattu
Red Hat
http://rajith.2rlabs.com/
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: cpp.testprofile failures
Posted by Rajith Attapattu <ra...@gmail.com>.
Martin,
I will try to create JIRA's and get you the logs.
Your help in finding the issues or fixing the tests is very much appreciated.
Regards,
Rajith
On Fri, Oct 23, 2009 at 3:31 AM, Martin Ritchie <ri...@apache.org> wrote:
> 2009/10/23 Rajith Attapattu <ra...@gmail.com>:
>> I have temporarily excluded the following tests from the cpp test
>> profiles to get them green.
>> All these tests have been added within the last 2 weeks.
>> I will investigate them one by one to ensure if any of the failures
>> are genuine or needs to be permanently disabled.
>>
>> Are these tests run against the 0-10 Java Broker?
>> If so I'd be interested to know the outcome.
>>
>> org.apache.qpid.test.unit.publish.DirtyTrasactedPubilshTest
>> org.apache.qpid.test.unit.ack.FailoverBeforeConsumingRecoverTest
>> org.apache.qpid.test.unit.ack.AcknowledgeAfterFailoverTest
>> org.apache.qpid.test.unit.ack.AcknowledgeAfterFailoverOnMessageTest
>> org.apache.qpid.test.client.RollbackOrderTest#testOrderingAfterRollbackOnMessage
>
> Rajith,
>
> I was working on the Ack tests earlier this week and believe there is
> still more to do.
> Currently as I see it there is at least one race condition within the client.
> The IO Thread that handles the Socket connection closure exception
> which then starts failover is causing our model
> AMQSession/AMQConnecton to be marked closed BEFORE failover has had a
> chance to work. As a result when failover completes and tries to use
> the Session it is closed so throws a JMSException.
>
> This is the cause of all the X/10 messages not received failures. The
> messages have been received but they were never delivered to the
> client because the preDelivery check has an operation that checks if
> the Session is closed. In this situation the connection is closed so a
> JMSException is thrown. I'm not 100% what happends to the mesasge. I
> think it just gets dropped.. not ideal. But as the Session can never
> be un-closed the session cannot be committed or acknowledged so
> message loss will not occur. The down side is that the app will appear
> not to receive messages and when the client is quit the messages will
> be released by the broker.
>
> I haven't seen the other tests you mention fail recently though I have
> recently been testing without the new IO Layer to try and nail down
> the client issues. The New IO layer seems to have a slightly different
> failure scenario which may be what you are seeing with the other
> tests.
>
> If you could JIRA and post some test run logs then perhaps I can help
> identify the issues.
>
> Cheers
>
> Martin
>
>
>> Regards,
>>
>> Rajith Attapattu
>> Red Hat
>> http://rajith.2rlabs.com/
>>
>> ---------------------------------------------------------------------
>> Apache Qpid - AMQP Messaging Implementation
>> Project: http://qpid.apache.org
>> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>>
>>
>
>
>
> --
> Martin Ritchie
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>
>
--
Regards,
Rajith Attapattu
Red Hat
http://rajith.2rlabs.com/
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: cpp.testprofile failures
Posted by Martin Ritchie <ri...@apache.org>.
2009/10/23 Rajith Attapattu <ra...@gmail.com>:
> I have temporarily excluded the following tests from the cpp test
> profiles to get them green.
> All these tests have been added within the last 2 weeks.
> I will investigate them one by one to ensure if any of the failures
> are genuine or needs to be permanently disabled.
>
> Are these tests run against the 0-10 Java Broker?
> If so I'd be interested to know the outcome.
>
> org.apache.qpid.test.unit.publish.DirtyTrasactedPubilshTest
> org.apache.qpid.test.unit.ack.FailoverBeforeConsumingRecoverTest
> org.apache.qpid.test.unit.ack.AcknowledgeAfterFailoverTest
> org.apache.qpid.test.unit.ack.AcknowledgeAfterFailoverOnMessageTest
> org.apache.qpid.test.client.RollbackOrderTest#testOrderingAfterRollbackOnMessage
Rajith,
I was working on the Ack tests earlier this week and believe there is
still more to do.
Currently as I see it there is at least one race condition within the client.
The IO Thread that handles the Socket connection closure exception
which then starts failover is causing our model
AMQSession/AMQConnecton to be marked closed BEFORE failover has had a
chance to work. As a result when failover completes and tries to use
the Session it is closed so throws a JMSException.
This is the cause of all the X/10 messages not received failures. The
messages have been received but they were never delivered to the
client because the preDelivery check has an operation that checks if
the Session is closed. In this situation the connection is closed so a
JMSException is thrown. I'm not 100% what happends to the mesasge. I
think it just gets dropped.. not ideal. But as the Session can never
be un-closed the session cannot be committed or acknowledged so
message loss will not occur. The down side is that the app will appear
not to receive messages and when the client is quit the messages will
be released by the broker.
I haven't seen the other tests you mention fail recently though I have
recently been testing without the new IO Layer to try and nail down
the client issues. The New IO layer seems to have a slightly different
failure scenario which may be what you are seeing with the other
tests.
If you could JIRA and post some test run logs then perhaps I can help
identify the issues.
Cheers
Martin
> Regards,
>
> Rajith Attapattu
> Red Hat
> http://rajith.2rlabs.com/
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>
>
--
Martin Ritchie
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: cpp.testprofile failures
Posted by Robert Godfrey <ro...@gmail.com>.
2009/10/23 Rajith Attapattu <ra...@gmail.com>
> I have temporarily excluded the following tests from the cpp test
> profiles to get them green.
> All these tests have been added within the last 2 weeks.
> I will investigate them one by one to ensure if any of the failures
> are genuine or needs to be permanently disabled.
>
> Are these tests run against the 0-10 Java Broker?
> If so I'd be interested to know the outcome.
>
> org.apache.qpid.test.unit.publish.DirtyTrasactedPubilshTest
> org.apache.qpid.test.unit.ack.FailoverBeforeConsumingRecoverTest
> org.apache.qpid.test.unit.ack.AcknowledgeAfterFailoverTest
> org.apache.qpid.test.unit.ack.AcknowledgeAfterFailoverOnMessageTest
>
> org.apache.qpid.test.client.RollbackOrderTest#testOrderingAfterRollbackOnMessage
>
>
>
Running the tests against the Java 0-10 broker, I get failures on
AcknowledgeAfterFailoverOnMessageTest (testTransacted,testNoAck ans
testPreAck when I ran it)... abd
AcknowledgeAfterFailoverTest<file:///home/rob/java-broker-0-10/qpid/java/build/report/org/apache/qpid/test/unit/ack/180_AcknowledgeAfterFailoverTest.html>(testPreAck)
... even on the InVM (0-8) tests I was getting race conditions
on these - I've fixed one case (where there was a deadlock because we
weren't checking if a connection was in the process of closing before
starting a send and taking the Failover lock)... but there seemed to be some
other intances where it would deadlock on connection close with Failover.
AFAICT nothing in this area had been changed by my 0-10 work... so I'm going
to assume that those race conditions are in trunk also...
-- Rob
> Regards,
>
> Rajith Attapattu
> Red Hat
> http://rajith.2rlabs.com/
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>
>