You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Robbie Gemmell (JIRA)" <ji...@apache.org> on 2015/06/23 11:44:02 UTC
[jira] [Resolved] (QPIDJMS-75) make the test peer report more
useful failure causes and update related tests to be more forgiving of
slowdowns on shared CI systems
[ https://issues.apache.org/jira/browse/QPIDJMS-75?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robbie Gemmell resolved QPIDJMS-75.
-----------------------------------
Resolution: Fixed
> make the test peer report more useful failure causes and update related tests to be more forgiving of slowdowns on shared CI systems
> ------------------------------------------------------------------------------------------------------------------------------------
>
> Key: QPIDJMS-75
> URL: https://issues.apache.org/jira/browse/QPIDJMS-75
> Project: Qpid JMS
> Issue Type: Task
> Affects Versions: 0.1.0, 0.2.0, 0.3.0
> Reporter: Robbie Gemmell
> Assignee: Robbie Gemmell
> Fix For: 0.4.0
>
>
> In adddition to running unit tests and some brokered interop/integration tests, we also created a 'test peer' to allow performing some brokerless integration testing of the client against a peer we could use to match/validate the AMQP frames emitted by the client and construct AMQP frames to send to the client.
> Thus far, whenever a particular matcher fails (e.g checking a particular frame field has an expected value, and it did not), the resulting assertion error in the test peer thread was recorded and then peer simply stops processing, wihtout undertaking any action that would have occurred had the value been as expected. As a result, if the client is awaiting a response from the peer (which in most cases it is) then it is never sent, and the test idles until it is timed out by JUnit (or ceased by some other action), which has also resulted in use of artificially low timesouts to bound run times against 'expected' failure cases, additionally making the tests unforgiving of sporadic slowdown on shared CI machines. Although the assertion failure is recorded, the test peer typically is not shut down in those cases as the test timeout itself becomes the reported cause of failure, leading to inspection of the test log output being necessary to identify anything about what actually happened.
> To improve things in terms of the reported test failure/error cause and also the time taken for the test to fail/error in many cases, assertion failures continue to be recorded but the subsequent actions actually still performed. Where the test is able to continue to completion the first assertion can then be thrown when closing down the test peer, meaning the test is much more likely to fail fast and the assertion failure actually becoming the reported cause/error by JUnit on the console, thus improving reporting and simplifying analysis. By avoiding need to use of the test timeout to bound run time in the case of these 'expected failures', the applied test timeouts can also be increased which enables the tests to be more forgiving of sporadic slowdowns while being run on shared CI machines.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org