RFD: Apache Regression Test Suite (ARTS) !!

["Ralf S. Engelschall" <rs...@engelschall.com>]

Yesterday I've tried to start the Apache Documentation Project (ADP) by
initially evualiting tools and then establishing the SGML skeleton an the
first content. Today I've tried to start another important project:

   The Apache Regression Test Suite (ARTS)

As discussed in the past, Apache needs such a test suite. So I've browsed
through my brain and knowledge again and established such a beasts runtime
engine end environment and the first two tests itself.

The idea is the following: 
   1. We use software leverage and take advantage of Perl's Test::Harness and
      LWP packages.
   2. We provide a Makefile and a bootstrapping script to be able to easily
      apply the test suite to a particular httpd.
   3. We provide utilitiy functions to make the individual tests as short as
      possible while providing as much flexibility we can.

The first result can be found on http://www.apache.org/~rse/. 
Grab the ARTS.tar.gz and unpack it anywhere. Then do

   $ cd ARTS
   $ make HTTPD=/path/to/our/apache/src/httpd

For me this looks like this:

| rse@en1:/e/apache/www/ARTS
| :> make HTTPD=/e/apache/test/src/httpd
| __ CONFIGURATION __
| Perl Interpreter (v5.004_01) [/sw/bin/perl]: 
| Apache httpd (Apache/1.3a2-dev): /e/apache/test/src/httpd
| 
| __ RUNNING TEST SUITE __
| t/01-basic..........ok
| t/02-get............ok
| All tests successful.
| Files=2,  Tests=3, 10 secs ( 1.95 cusr  0.30 csys =  2.24 cpu)
| rse@en1:/e/apache/www/ARTS
| :>

That's all folks. Currently there are 3 simple tests just to see if the
approach works as expected.

Feedback?
                                       Ralf S. Engelschall
                                       rse@engelschall.com
                                       www.engelschall.com

Re: RFD: Apache Regression Test Suite (ARTS) !!

[Dean Gaudet <dg...@arctic.org>]

On Tue, 5 Aug 1997, Ralf S. Engelschall wrote:

>    1. We use software leverage and take advantage of Perl's Test::Harness and
>       LWP packages.

This lets you do basic tests of config functionality ... but how easy
would it be to verify protocol correctness?  I imagine that's as easy as
using one of the chat packages and parsing results more verbosely.  Can
LWP issue 1.1 requests? 

Otherwise this is a great approach. 

Dean

Re: RFD: Apache Regression Test Suite (ARTS) !!

[Marc Slemko <ma...@worldgate.com>]

On Tue, 5 Aug 1997, Ralf S. Engelschall wrote:

> 
> Yesterday I've tried to start the Apache Documentation Project (ADP) by
> initially evualiting tools and then establishing the SGML skeleton an the
> first content. Today I've tried to start another important project:
> 
>    The Apache Regression Test Suite (ARTS)

So, what are you planning for tomorrow? <g>

> 
> As discussed in the past, Apache needs such a test suite. So I've browsed
> through my brain and knowledge again and established such a beasts runtime
> engine end environment and the first two tests itself.
> 
> The idea is the following: 
>    1. We use software leverage and take advantage of Perl's Test::Harness and
>       LWP packages.
>    2. We provide a Makefile and a bootstrapping script to be able to easily
>       apply the test suite to a particular httpd.
>    3. We provide utilitiy functions to make the individual tests as short as
>       possible while providing as much flexibility we can.

Exactly what sort of tests are we talking about?  It is not a bad idea,
but I am wary of the work it would take to establish a usefully complete
set of tests.