Testing setup changes

[Andrew Black <ab...@roguewave.com>]

Greetings all.

Earlier today I checked a change into our internal version control 
system that will alter how we do our nightly testing of stdcxx.  This 
change alters the the sources used in the testing to include our 
internal (legacy) tests and test driver.  As a result, a couple things 
will be noticed.

First, you will see something around 100 additional tests.  These are 
tests that still need to be ported from the internal testing framework 
to the rwtest driver.  These tests can be distinguished from the test 
suite tests by the names used.  The legacy tests are either named in the 
form N_whatver, or lack a numeric test group identifier.

Second, until some macromagic can be fixed, the rwtest library on 
windows will fail to build, due to issues in the legacy test framwork, 
which is compiled into the rwtest library.  There are a couple ways 
these issues can be resolved, and we will try to resolve them as soon as 
we can.  This build failure will only be observed in nightly testing, as 
the failing source files are part of the legacy test framework, and 
aren't included in subversion.

--Andrew Black

Re: Testing setup changes

[Martin Sebor <se...@roguewave.com>]

Andrew Black wrote:
> Greetings all.
> 
> Earlier today I checked a change into our internal version control 
> system that will alter how we do our nightly testing of stdcxx.  This 
> change alters the the sources used in the testing to include our 
> internal (legacy) tests and test driver.

This has been deployed:
   http://people.apache.org/~sebor/stdcxx/results/

Some of the results are kind of flakey right now and not all
the most recent ones correspond to the recent state of trunk.
In particular, many (but not all) of the red rows are out of
date and the problems they represent have already been fixed.

Andrew's working on stabilizing things and we hope to have
more reliable data on the page early next week.

Martin

> As a result, a couple things 
> will be noticed.
> 
> First, you will see something around 100 additional tests.  These are 
> tests that still need to be ported from the internal testing framework 
> to the rwtest driver.  These tests can be distinguished from the test 
> suite tests by the names used.  The legacy tests are either named in the 
> form N_whatver, or lack a numeric test group identifier.
> 
> Second, until some macromagic can be fixed, the rwtest library on 
> windows will fail to build, due to issues in the legacy test framwork, 
> which is compiled into the rwtest library.  There are a couple ways 
> these issues can be resolved, and we will try to resolve them as soon as 
> we can.  This build failure will only be observed in nightly testing, as 
> the failing source files are part of the legacy test framework, and 
> aren't included in subversion.
> 
> --Andrew Black