You are viewing a plain text version of this content. The canonical link for it is here.
Posted to test-dev@httpd.apache.org by Rodent of Unusual Size <Ke...@Golux.Com> on 2002/01/02 19:57:51 UTC
Re: perl-framework server startup
Was this feedback useful, Stas?
--
#ken P-)}
Ken Coar, Sanagendamgagwedweinini http://Golux.Com/coar/
Author, developer, opinionist http://Apache-Server.Com/
"All right everyone! Step away from the glowing hamburger!"
[patch] (was Re: perl-framework server startup)
Posted by Stas Bekman <st...@stason.org>.
On Thu, 3 Jan 2002, Stas Bekman wrote:
> Rodent of Unusual Size wrote:
>
> > Was this feedback useful, Stas?
>
> Yes, yes. I can reproduce the problem. The respawned shell to set the
> ulimit loses the return status. I'm looking into it. It's a nasty one.
OK, Ken, please try this patch. If it works for you please try it in
various possible failure situations so we can close this issue. Thanks.
this patch changes two "things":
1. uses exec() to call itself for setting ulimit (this solves the lost
status problem). Hope this is portable.
2. directs all exit() calls in PerlRun.pm into one place for two reasons.
- Enable easier debug in the future
- functions like server->stop don't return 0/1 but -1..N, so it helps to
handle the exit arguments properly.
in addition all exit() calls ends in exit_shell, to which you may want to
pass a real return status which can have quite a few values.
Index: Apache-Test/lib/Apache/TestRun.pm
===================================================================
RCS file: /home/cvs/httpd-test/perl-framework/Apache-Test/lib/Apache/TestRun.pm,v
retrieving revision 1.80
diff -u -r1.80 TestRun.pm
--- Apache-Test/lib/Apache/TestRun.pm 31 Dec 2001 09:09:43 -0000 1.80
+++ Apache-Test/lib/Apache/TestRun.pm 3 Jan 2002 12:27:49 -0000
@@ -17,6 +17,7 @@
use Config;
use constant STARTUP_TIMEOUT => 300; # secs (good for extreme debug cases)
+use subs qw(exit_shell exit_perl);
my %core_files = ();
@@ -137,7 +138,7 @@
my @invalid_argv = @{ $self->{argv} };
if (@invalid_argv) {
error "unknown opts or test names: @invalid_argv";
- exit;
+ exit_perl 0;
}
}
@@ -258,16 +259,17 @@
return unless $_[0] =~ /^Failed/i; #dont catch Test::ok failures
$server->stop(1) if $opts->{'start-httpd'};
$server->failed_msg("error running tests");
+ exit_perl 0;
};
$SIG{INT} = sub {
if ($caught_sig_int++) {
warning "\ncaught SIGINT";
- exit;
+ exit_perl 0;
}
warning "\nhalting tests";
$server->stop if $opts->{'start-httpd'};
- exit;
+ exit_perl 0;
};
#try to make sure we scan for core no matter what happens
@@ -383,17 +385,19 @@
for (@exit_opts) {
next unless exists $self->{opts}->{$_};
my $method = "opt_$_";
- exit if $self->$method();
+ exit_perl $self->$method();
}
if ($self->{opts}->{'stop-httpd'}) {
+ my $ok = 1;
if ($self->{server}->ping) {
- $self->{server}->stop;
+ $ok = $self->{server}->stop;
+ $ok = $ok < 0 ? 0 : 1; # adjust to 0/1 logic
}
else {
warning "server $self->{server}->{name} is not running";
}
- exit;
+ exit_perl $ok ;
}
}
@@ -407,7 +411,7 @@
($test_config->{APXS} ?
"an apxs other than $test_config->{APXS}" : "apxs").
" or put either in your PATH";
- exit 1;
+ exit_perl 0;
}
my $opts = $self->{opts};
@@ -427,7 +431,7 @@
}
if ($opts->{'start-httpd'}) {
- exit 1 unless $server->start;
+ exit_perl 0 unless $server->start;
}
elsif ($opts->{'run-tests'}) {
my $is_up = $server->ping
@@ -436,7 +440,7 @@
&& $server->wait_till_is_up(STARTUP_TIMEOUT));
unless ($is_up) {
error "server is not ready yet, try again.";
- exit;
+ exit_perl 0;
}
}
}
@@ -464,7 +468,7 @@
sub stop {
my $self = shift;
- $self->{server}->stop if $self->{opts}->{'stop-httpd'};
+ return $self->{server}->stop if $self->{opts}->{'stop-httpd'};
}
sub new_test_config {
@@ -491,13 +495,10 @@
}
close $sh;
- open $sh, "|$binsh" or die;
- my @cmd = ("ulimit -c unlimited\n",
- "exec $0 @ARGV");
- warning "setting ulimit to allow core files\n@cmd";
- print $sh @cmd;
- close $sh;
- exit; #exec above will take over
+ my $command = "ulimit -c unlimited; $0 @ARGV";
+ warning "setting ulimit to allow core files\n$command";
+ exec $command;
+ die "exec $command has failed"; # shouldn't be reached
}
sub set_ulimit {
@@ -548,13 +549,13 @@
warning "forcing Apache::TestConfig object save";
$self->{test_config}->save;
warning "run 't/TEST -clean' to clean up before continuing";
- exit 1;
+ exit_perl 0;
}
}
if ($self->{opts}->{configure}) {
warning "reconfiguration done";
- exit;
+ exit_perl 1;
}
$self->try_exit_opts;
@@ -770,5 +771,18 @@
}
+# in idiomatic perl functions return 1 on success 0 on
+# failure. Shell expects the opposite behavior. So this function
+# reverses the status.
+sub exit_perl {
+ exit_shell $_[0] ? 0 : 1;
+}
+
+# expects shell's exit status values (0==success)
+sub exit_shell {
+# require Carp;
+# Carp::cluck('exiting');
+ CORE::exit $_[0];
+}
1;
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: perl-framework server startup
Posted by Stas Bekman <st...@stason.org>.
Rodent of Unusual Size wrote:
> Stas Bekman wrote:
>
>>Of course, I thought that's what you want, so you can do a binary and
>>and figure out what has failed.
>>
>
> Okey, how do a bitwise AND in the shell in a portable way?
>
>
>>OK, but harness may fail for different reasons, why do you want to
>>scratch the status code of the failure? Currently you don't care, but
>>someone may want to know the failure status and program the launching
>>program to act accordingly.
>>
>
> Fine. That's an excellent point. Now, if only I can figure out
> a better way to test this than
>
> STATUS=$?
> if perl -e "exit (($STATUS & 0xc8) == 0xc8);" ; then
> harness_failed
> elif test "x$STATUS" != "x0" ; then
> test_failed
> fi
Sorry, I know a little csh programming, much less sh, but I do know Perl :)
#!/usr/bin/perl
use constant TEST_FAILED => 0xC8;
test(q{perl -e 'exit 0xC9'});
test(q{perl -e 'exit 0x01'});
sub test {
my $command = shift;
system $command;
$status = $? >> 8;
print $status & TEST_FAILED ? "Test\n" : "Harness\n";
}
I realize that you want to do this in pure sh, but then I guess you need
to lookup how to perform binary logic in sh. On the other hand why not
to code your the scripts in Perl in first place? So much easier...
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: perl-framework server startup
Posted by Rodent of Unusual Size <Ke...@Golux.Com>.
Stas Bekman wrote:
>
> Of course, I thought that's what you want, so you can do a binary and
> and figure out what has failed.
Okey, how do a bitwise AND in the shell in a portable way?
> OK, but harness may fail for different reasons, why do you want to
> scratch the status code of the failure? Currently you don't care, but
> someone may want to know the failure status and program the launching
> program to act accordingly.
Fine. That's an excellent point. Now, if only I can figure out
a better way to test this than
STATUS=$?
if perl -e "exit (($STATUS & 0xc8) == 0xc8);" ; then
harness_failed
elif test "x$STATUS" != "x0" ; then
test_failed
fi
:-)
--
#ken P-)}
Ken Coar, Sanagendamgagwedweinini http://Golux.Com/coar/
Author, developer, opinionist http://Apache-Server.Com/
"Millenium hand and shrimp!"
Re: perl-framework server startup
Posted by Stas Bekman <st...@stason.org>.
Rodent of Unusual Size wrote:
> * On 2002-01-21 at 21:41,
> Stas Bekman <st...@stason.org> excited the electrons to say:
>
>>OK, be it 0xC8 . Try this patch then:
>>
>>Index: Apache-Test/lib/Apache/TestRun.pm
>>===================================================================
>>
> :
>
>>+ # logically OR 0xff to the status, so we can distinguish
>>+ # harness failure from tests failure. it's possible that the
>>+ # failing test won't set a proper $?
>>+ exit_shell ($?||1) | TEST_FAILED_MASK;
>>
>
> Er, unless I'm missing something (entirely possible), this will
> never return 0xc8.. If $? == 0..1, it'll return 0xc9.
Of course, I thought that's what you want, so you can do a binary and
and figure out what has failed.
BTW, this comment is wrong, should be:
# Binary OR'ing with exit_status with TEST_FAILED_MASK, so
# when the program exits one can tell test failures from
# harness failures if 'exit_status && TEST_FAILED_MASK' is
# true.
> I didn't mean
> use 0xc8 as a *mask* (since that makes shell manipulation of the
> bits a pain), but as a value:
>
> if failed(harness) {
> exit(0xc8)
> }
> else {
> exit(whatever test status was)
> }
>
> If the harness fails to run properly, I doubt any test
> results are going to be meaningful, so an OR mask doesn't
> seem useful. I proposed 0xc8 (200d) because it didn't look
> as though anything else could return it.
OK, but harness may fail for different reasons, why do you want to
scratch the status code of the failure? Currently you don't care, but
someone may want to know the failure status and program the launching
program to act accordingly.
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: perl-framework server startup
Posted by Rodent of Unusual Size <Ke...@Golux.Com>.
* On 2002-01-21 at 21:41,
Stas Bekman <st...@stason.org> excited the electrons to say:
>
> OK, be it 0xC8 . Try this patch then:
>
> Index: Apache-Test/lib/Apache/TestRun.pm
> ===================================================================
:
> + # logically OR 0xff to the status, so we can distinguish
> + # harness failure from tests failure. it's possible that the
> + # failing test won't set a proper $?
> + exit_shell ($?||1) | TEST_FAILED_MASK;
Er, unless I'm missing something (entirely possible), this will
never return 0xc8.. If $? == 0..1, it'll return 0xc9. I didn't mean
use 0xc8 as a *mask* (since that makes shell manipulation of the
bits a pain), but as a value:
if failed(harness) {
exit(0xc8)
}
else {
exit(whatever test status was)
}
If the harness fails to run properly, I doubt any test
results are going to be meaningful, so an OR mask doesn't
seem useful. I proposed 0xc8 (200d) because it didn't look
as though anything else could return it.
--
#ken P-)}
Ken Coar, Sanagendamgagwedweinini http://Golux.Com/coar/
Author, developer, opinionist http://Apache-Server.Com/
"Millenium hand and shrimp!"
Re: perl-framework server startup
Posted by Stas Bekman <st...@stason.org>.
Rodent of Unusual Size wrote:
> Stas Bekman wrote:
>
>>I've played with the returned status, so here is what I saw.
>>
>>If you want to tell shell that the program has failed, you must
>>return a status with at least one bit in the 0x01-0xff range set.
>>
>
> Yar, looking at 'man bash' and empirical results, it looks as
> though the exit value of a command is always ANDed with 0xff.
> And (for bash at least) an unhandled signal exit will result in
> an exit code of (0x80 + signum). For Linux, the maximum
> signal number is 63, so exit values in the range of 0x81-0xbf
> are reserved for signals.
>
> So why don't we use an exit value of 200 to indicate a harness
> failure, and anything else comes from the tests themselves?
> This should also keep us clear of any untrapped die() calls,
> if I'm understanding the mechanism correctly.
OK, be it 0xC8 . Try this patch then:
Index: Apache-Test/lib/Apache/TestRun.pm
===================================================================
RCS file:
/home/cvs/httpd-test/perl-framework/Apache-Test/lib/Apache/TestRun.pm,v
retrieving revision 1.85
diff -u -r1.85 TestRun.pm
--- Apache-Test/lib/Apache/TestRun.pm 16 Jan 2002 17:05:20 -0000 1.85
+++ Apache-Test/lib/Apache/TestRun.pm 22 Jan 2002 01:59:40 -0000
@@ -17,6 +17,10 @@
use Config;
use constant STARTUP_TIMEOUT => 300; # secs (good for extreme debug cases)
+
+# do 'exit_status && 200' to tell test failures from harness failures
+use constant TEST_FAILED_MASK => 0xC8;
+
use subs qw(exit_shell exit_perl);
my %core_files = ();
@@ -258,11 +262,17 @@
my($server, $opts) = ($self->{server}, $self->{opts});
+ # this handler gets usually invoked when one or more tests fail
+ # in Test::Harness::runtests()
$SIG{__DIE__} = sub {
return unless $_[0] =~ /^Failed/i; #dont catch Test::ok failures
$server->stop(1) if $opts->{'start-httpd'};
$server->failed_msg("error running tests");
- exit_perl 0;
+
+ # logically OR 0xff to the status, so we can distinguish
+ # harness failure from tests failure. it's possible that the
+ # failing test won't set a proper $?
+ exit_shell ($?||1) | TEST_FAILED_MASK;
};
$SIG{INT} = sub {
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: perl-framework server startup
Posted by Rodent of Unusual Size <Ke...@Golux.Com>.
Stas Bekman wrote:
>
> I've played with the returned status, so here is what I saw.
>
> If you want to tell shell that the program has failed, you must
> return a status with at least one bit in the 0x01-0xff range set.
Yar, looking at 'man bash' and empirical results, it looks as
though the exit value of a command is always ANDed with 0xff.
And (for bash at least) an unhandled signal exit will result in
an exit code of (0x80 + signum). For Linux, the maximum
signal number is 63, so exit values in the range of 0x81-0xbf
are reserved for signals.
So why don't we use an exit value of 200 to indicate a harness
failure, and anything else comes from the tests themselves?
This should also keep us clear of any untrapped die() calls,
if I'm understanding the mechanism correctly.
--
#ken P-)}
Ken Coar, Sanagendamgagwedweinini http://Golux.Com/coar/
Author, developer, opinionist http://Apache-Server.Com/
"Millenium hand and shrimp!"
Re: perl-framework server startup
Posted by Stas Bekman <st...@stason.org>.
On Wed, 16 Jan 2002, Rodent of Unusual Size wrote:
> Stas Bekman wrote:
> >
> > But which bit is safe to OR with? 0x10 or 0x1000 as you've suggested?
>
> I would say 0x?000 -- values in the 0x01-0xff range get set from signals
> and errno, n'est-�e pas?
I've played with the returned status, so here is what I saw.
If you want to tell shell that the program has failed, you must return a
status with at least one bit in the 0x01-0xff range set. But if you set
any bits on the higher bits, it seems that shell masks these off. So I'm
not sure if you ever get to see them.
Assuming that modperl/readline test fails (without setting $? from the
test, so the default 1 is applied) and I run:
% t/TEST -v modperl/readline || echo "Failed $?"
I get:
Failed 1
and no high bits here. This is with the patch below. Please play with this
patch and tell me whether it does what you want, and if not how we should
change it.
Index: Apache-Test/lib/Apache/TestRun.pm
===================================================================
RCS file: /home/cvs/httpd-test/perl-framework/Apache-Test/lib/Apache/TestRun.pm,v
retrieving revision 1.85
diff -u -r1.85 TestRun.pm
--- Apache-Test/lib/Apache/TestRun.pm 16 Jan 2002 17:05:20 -0000 1.85
+++ Apache-Test/lib/Apache/TestRun.pm 17 Jan 2002 03:34:05 -0000
@@ -258,11 +258,17 @@
my($server, $opts) = ($self->{server}, $self->{opts});
+ # this handler gets usually invoked when one or more tests fail
+ # in Test::Harness::runtests()
$SIG{__DIE__} = sub {
return unless $_[0] =~ /^Failed/i; #dont catch Test::ok failures
$server->stop(1) if $opts->{'start-httpd'};
$server->failed_msg("error running tests");
- exit_perl 0;
+
+ # logically OR 0x1000 to the status, so we can distinguish
+ # harness failure from tests failure. it's possible that the
+ # failing test won't set a proper $?
+ exit_shell ( $? || 1 ) | 0x1000 ;
};
$SIG{INT} = sub {
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: perl-framework server startup
Posted by Rodent of Unusual Size <Ke...@Golux.Com>.
Stas Bekman wrote:
>
> But which bit is safe to OR with? 0x10 or 0x1000 as you've suggested?
I would say 0x?000 -- values in the 0x01-0xff range get set from signals
and errno, n'est-çe pas?
--
#ken P-)}
Ken Coar, Sanagendamgagwedweinini http://Golux.Com/coar/
Author, developer, opinionist http://Apache-Server.Com/
"Millenium hand and shrimp!"
Re: perl-framework server startup
Posted by Stas Bekman <st...@stason.org>.
Rodent of Unusual Size wrote:
> Stas Bekman wrote:
>
>>There may be many reasons for harness failure, so it seems to me that
>>it'll be much easier to have a special failure status code for failing
>>tests, no?
>>
>
> Sí.
But which bit is safe to OR with? 0x10 or 0x1000 as you've suggested?
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: perl-framework server startup
Posted by Rodent of Unusual Size <Ke...@Golux.Com>.
Stas Bekman wrote:
>
> There may be many reasons for harness failure, so it seems to me that
> it'll be much easier to have a special failure status code for failing
> tests, no?
Sí.
--
#ken P-)}
Ken Coar, Sanagendamgagwedweinini http://Golux.Com/coar/
Author, developer, opinionist http://Apache-Server.Com/
"All right everyone! Step away from the glowing hamburger!"
Re: perl-framework server startup
Posted by Stas Bekman <st...@stason.org>.
Rodent of Unusual Size wrote:
> Here I come, being a pain again..
>
> I'd like to be able to differentiate between a failure
> of some tests and failure of the harness. That is,
> if some of the tests failed, that may be expected --
> but if the harness fails, I want to be able to say
> 'whoa.'
>
> This turned up in last night's run of my autotester,
> which now displays the t/logs/error_log if it detects
> a error exit code from t/TEST. The harness ran just
> fine, but some of the tests failed -- so I got the
> 2MB error_log even though I didn't want it. I [currently]
> only want to see the error_log if the *harness* failed.
>
> Suggestions? Maybe ORing the status with some value
> (like 0x1000) if the failure was in the harness itself
> (e.g., unable to start the server)?
There may be many reasons for harness failure, so it seems to me that
it'll be much easier to have a special failure status code for failing
tests, no?
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: perl-framework server startup
Posted by Rodent of Unusual Size <Ke...@Golux.Com>.
Here I come, being a pain again..
I'd like to be able to differentiate between a failure
of some tests and failure of the harness. That is,
if some of the tests failed, that may be expected --
but if the harness fails, I want to be able to say
'whoa.'
This turned up in last night's run of my autotester,
which now displays the t/logs/error_log if it detects
a error exit code from t/TEST. The harness ran just
fine, but some of the tests failed -- so I got the
2MB error_log even though I didn't want it. I [currently]
only want to see the error_log if the *harness* failed.
Suggestions? Maybe ORing the status with some value
(like 0x1000) if the failure was in the harness itself
(e.g., unable to start the server)?
--
#ken P-)}
Ken Coar, Sanagendamgagwedweinini http://Golux.Com/coar/
Author, developer, opinionist http://Apache-Server.Com/
"All right everyone! Step away from the glowing hamburger!"
Re: perl-framework server startup
Posted by Stas Bekman <st...@stason.org>.
Rodent of Unusual Size wrote:
> Was this feedback useful, Stas?
Yes, yes. I can reproduce the problem. The respawned shell to set the
ulimit loses the return status. I'm looking into it. It's a nasty one.
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/