Re: cvs commit: apr-util/test CuTest.c CuTest.h test_aprutil.h testall.c .cvsignore Makefile.in testuuid.c

[rb...@rkbloom.net]

Quoting rbb@apache.org:

> rbb         2004/03/10 18:42:34
> 
>   Modified:    test     .cvsignore Makefile.in testuuid.c
>   Added:       test     CuTest.c CuTest.h test_aprutil.h testall.c
>   Log:
>   Add the test framework from APR to APR-util, and port testuuid to the
>   new framework.

As soon as more of the APR-util test suite is ported to testall, I will add the
coverage scripts that APR is using now.

My reason for moving to the unified test suite is very simple.  I have run the
original tests from APR and APR-util, and I can't look at the output and know if
the tests passed or not.  The original tests from APR used a simple format that
I thought was good enough when I first created the tests.  It was basically:

test description ............................ PASS | FAIL

This turns out to be almost good enough to a degree, because it is easy to tell
if everything passed or not at a glance.  But, it fails as soon as the test
suite gets large, because you have to scroll through pages of data, and the FAIL
response can easily get lost in the noise.  There is also a lot of noise,
because you don't really care what the exact tests are that are being run unless
a test fails.  

The unified test suite has a much simpler format:

test class:  results

results is a string of '.', 'F', or 'N'.  '.' means pass, 'F' means fail, and
'N' means not implemented.  There is one character per test.  This makes it
trivial to determine if a test has passed or failed.  If the test fails, you can
get the details by running ./testall -v.

There are a couple of problems with this test suite that I am currently in the
middle of fixing, but it will probably take me a couple of weeks to fully
finish.  The first problem is that we have a character for each test.  In the
future, there will be an indicator that the tests are running, and then there
will be a result string that looks like:  "X of Y tests FAILED".  Now you are
guaranteed one line per class of test (File, network, uuid, etc).  Another
problem is that the test suite basically runs all of the tests in the class and
then prints the results.  This will be changed.  In the future, the test suite
will print the class name, then print the indicator, and when all tests are done
it will print the results.

The final problem is that parts of the test suite API need work.  The biggest
problem is that you can't pass arbitrary data to the test function itself.  So,
we have some tests that iterate through an array of data.  Each iteration should
be a test in itself, but the test suite can't do that easily.  So, I will be
adding that feature.

If anybody disagrees with my logic behind changing the test suites, please speak
up, I have no problem at all not using this test suite, but I do have a problem
having a set of tests that I can't easily translate into success or failure.

Ryan