[Fwd: FAIL Apache-Test-1.06 darwin 6.8]

[Geoffrey Young <ge...@modperlcookbook.org>]

hi all

I've been thinking about this situation for a while now...

-------- Original Message --------
Subject: FAIL Apache-Test-1.06 darwin 6.8
Date: Tue, 11 Nov 2003 01:03:19 +0200 (IST)

!!! no test server configured, please specify an httpd or apxs or put either 
in your PATH. For example:
t/TEST -httpd /path/to/bin/httpd
+-----------------------------------------------------+
| To report problems please refer to the SUPPORT file |
+-----------------------------------------------------+
make: *** [run_tests] Error 1


obviously, the tests are not failing - the test suite is only partially (and 
thus improperly) configured.  so, it bothers me somewhat that 'make test' 
reports back a failure when the package didn't fail, it merely failed to run 
the test suite.

in some work I've been doing lately, I go to great lengths to skip 
Apache-Test setup if there are no -apxs, -httpd, or proper %ENV settings 
when the Makefile is generated, which is particularly important if 
Apache-Test is only relevant for a small portion of the tests.

so, I'm considering changing the situation to have 'make test' return true 
if (and only if) no apache server can be located.  configuration problems, 
etc, will still mean failure.

yes, I know that

$ make && make test && sudo make install

will still install the package, even though the tests didn't run, I'm just 
not convinced that is wrong here - it's not a test failure if the tests 
can't run, and we often "pass" tests when we don't have the proper 
environment (plan skip_all => have_module('foo')).  this is just an 
extension of the same idea.

I would appreciate it if everyone listening could weigh in on this issue (so 
it's not just a debate between two people who probably already don't agree 
:)  I wouldn't be terribly upset if it stayed as it is currently, but my 
position on what tests should be is shifting and I'm more and more convinced 
that this approach is the one to take.

so (constructive) feedback welcome.

--Geoff

Re: [Fwd: FAIL Apache-Test-1.06 darwin 6.8]

[Stas Bekman <st...@stason.org>]

Geoffrey Young wrote:
> hi all
> 
> I've been thinking about this situation for a while now...

Hehe, the joys of releasing A-T ;)

> -------- Original Message --------
> Subject: FAIL Apache-Test-1.06 darwin 6.8
> Date: Tue, 11 Nov 2003 01:03:19 +0200 (IST)
> 
> !!! no test server configured, please specify an httpd or apxs or put 
> either in your PATH. For example:
> t/TEST -httpd /path/to/bin/httpd
> +-----------------------------------------------------+
> | To report problems please refer to the SUPPORT file |
> +-----------------------------------------------------+
> make: *** [run_tests] Error 1
> 
> 
> obviously, the tests are not failing - the test suite is only partially 
> (and thus improperly) configured.  so, it bothers me somewhat that 'make 
> test' reports back a failure when the package didn't fail, it merely 
> failed to run the test suite.
> 
> in some work I've been doing lately, I go to great lengths to skip 
> Apache-Test setup if there are no -apxs, -httpd, or proper %ENV settings 
> when the Makefile is generated, which is particularly important if 
> Apache-Test is only relevant for a small portion of the tests.
> 
> so, I'm considering changing the situation to have 'make test' return 
> true if (and only if) no apache server can be located.  configuration 
> problems, etc, will still mean failure.
> 
> yes, I know that
> 
> $ make && make test && sudo make install
> 
> will still install the package, even though the tests didn't run, I'm 
> just not convinced that is wrong here - it's not a test failure if the 
> tests can't run, and we often "pass" tests when we don't have the proper 
> environment (plan skip_all => have_module('foo')).  this is just an 
> extension of the same idea.
> 
> I would appreciate it if everyone listening could weigh in on this issue 
> (so it's not just a debate between two people who probably already don't 
> agree :)  I wouldn't be terribly upset if it stayed as it is currently, 
> but my position on what tests should be is shifting and I'm more and 
> more convinced that this approach is the one to take.
> 
> so (constructive) feedback welcome.

First of all I'm a solid -1 on making a failing 'make test' automatically 
returning success if its config has failed.

Here is the technical explanation for this vote is:

If you want to add this as an option that's cool, so developers could say, I 
don't care if people don't run my carefully designed coverage test suite and 
then bug me with problems in their programs which they won't have happened if 
they did run the test suite and reported the problems with a nicely crafted 
tracing and debug output, instead wasting my and user's time in an endless 
ping pong of emails... I definitely don't want to enable this on my modules 
and won't suggest to any other developers to do so. The SUPPORT doc should 
explain what to do in that case without causing a bug report (see more below).

The solution as I see it is as follows:

1. ignore those bugreporters who don't bother to read that error message. Yes, 
just ignore them, delete their (usually automatic via RT) email as soon as you 
see it. Since we now print the banner suggesting to look in SUPPORT on 
failure, we should probably explain at the beginning of that file what the 
error message means and what to do before trying to file a bug report. Also we 
should probably add the -apxs hint as well (before the existing -httpd)

2. add the configure-once run-all CPAN.pm-like feature, which Randy coded the 
first pass, I started to adopt it to use CPAN/FirstTime.pm but was sidetracked 
by nasty segfaults which i still haven't resolved. Once I finish with them I 
promise to take care of this issue as the first priority. Once this is 
accomplished, most people will need to tell -apxs manually once in their long 
life of using mp1 or mp2.

__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com