Latest passing build fails unit tests

[Chester <kn...@gmail.com>]

I noticed this build:
https://builds.apache.org/blue/organizations/jenkins/Qpid-cpp-trunk-test/detail/Qpid-cpp-trunk-test/1284/pipeline
is marked as successful.

The build log says:

*91% tests passed, 2 tests failed out of 22*

*Total Test time (real) = 2508.89 sec*

*The following tests FAILED:*
*   4 - acl_tests (Timeout)*
* 22 - sasl_tests (Failed)*
*Errors while running CTest*
*+ exit 0*
*[WARNINGS] Parsing warnings in console log with parser GNU Make + GNU C
Compiler (gcc)Skipping warnings blame since Git is the only supported SCM
up to now.%n[WARNINGS] Computing warning deltas based on reference build
#1283INFO: Starting to record.INFO: Processing CTest-Version 3.x
(default)INFO: [CTest-Version 3.x (default)] - 1 test report file(s) were
found with the pattern 'cpp-build/Testing/**/*.xml' relative to
'/home/jenkins/jenkins-slave/workspace/Qpid-cpp-trunk-test' for the testing
framework 'CTest-Version 3.x (default)'.INFO: Check 'Failed Tests'
threshold.INFO: Check 'Skipped Tests' threshold.INFO: Setting the build
status to SUCCESSINFO: Stopping recording.Finished: SUCCESS*


Looks like the unit tests aren't actually passing, but the build is marked
successful. Is this by intent?

Thanks,
Chester

Re: Latest passing build fails unit tests

[Gordon Sim <gs...@redhat.com>]

On 12/10/18 16:22, Chester wrote:
> I noticed this build:
> https://builds.apache.org/blue/organizations/jenkins/Qpid-cpp-trunk-test/detail/Qpid-cpp-trunk-test/1284/pipeline
> is marked as successful.
> 
> The build log says:
> 
> *91% tests passed, 2 tests failed out of 22*
> 
> *Total Test time (real) = 2508.89 sec*
> 
> *The following tests FAILED:*
> *   4 - acl_tests (Timeout)*
> * 22 - sasl_tests (Failed)*
> *Errors while running CTest*
> *+ exit 0*
> *[WARNINGS] Parsing warnings in console log with parser GNU Make + GNU C
> Compiler (gcc)Skipping warnings blame since Git is the only supported SCM
> up to now.%n[WARNINGS] Computing warning deltas based on reference build
> #1283INFO: Starting to record.INFO: Processing CTest-Version 3.x
> (default)INFO: [CTest-Version 3.x (default)] - 1 test report file(s) were
> found with the pattern 'cpp-build/Testing/**/*.xml' relative to
> '/home/jenkins/jenkins-slave/workspace/Qpid-cpp-trunk-test' for the testing
> framework 'CTest-Version 3.x (default)'.INFO: Check 'Failed Tests'
> threshold.INFO: Check 'Skipped Tests' threshold.INFO: Setting the build
> status to SUCCESSINFO: Stopping recording.Finished: SUCCESS*
> 
> 
> Looks like the unit tests aren't actually passing, but the build is marked
> successful. Is this by intent?

My guess is it is not by intent.

It looks like the tests both fail due to not finding the saslpasswd2 
tool used to setup the test database. The logs have:

   "Skipping SASL test, saslpasswd2 not found"

However that is actually only skipping the setup of the user database 
against which the tests would then authenticate. The ssl test itself is 
only skipped if cyrus sasl support was not built. The acl test doesn't 
have any checking from what I can see.

I propose the following change to fix the checking of the tests:

   https://reviews.apache.org/r/69025/

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org