You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spamassassin.apache.org by ms...@apache.org on 2004/11/25 21:45:33 UTC
svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Author: mss
Date: Thu Nov 25 12:45:32 2004
New Revision: 106600
URL: http://svn.apache.org/viewcvs?view=rev&rev=106600
Log:
What's the probability that I run into an already used port with the new probably_unused_spamd_port() code? Less than 1 per mill? Ask Murphy...
The routine now tries to ask netstat if that port is already in use. I tested the pattern on Linux, FreeBSD and Windows. If netstat can't be run, no harm is done, the routine will just work as before. The grep is pretty broad, it might also catch a remote port; then it just tries the next random one. (Hey Murphy, it really can't hit a used port ten times, can it?)
Modified:
spamassassin/trunk/t/SATest.pm
Modified: spamassassin/trunk/t/SATest.pm
Url: http://svn.apache.org/viewcvs/spamassassin/trunk/t/SATest.pm?view=diff&rev=106600&p1=spamassassin/trunk/t/SATest.pm&r1=106599&p2=spamassassin/trunk/t/SATest.pm&r2=106600
==============================================================================
--- spamassassin/trunk/t/SATest.pm (original)
+++ spamassassin/trunk/t/SATest.pm Thu Nov 25 12:45:32 2004
@@ -133,10 +133,15 @@
# suite runs on the same machine simultaneously
sub probably_unused_spamd_port {
my $port;
+ my @nstat = ();
+ if (open(NSTAT, "netstat -a -n 2>&1 |")) {
+ @nstat = grep(/^\s*tcp/i, <NSTAT>);
+ close(NSTAT);
+ }
my $delta = ($$ % 32768) || int(rand(32768));
for (1..10) {
$port = 32768 + $delta;
- last unless getservbyport($port, "tcp");
+ last unless (getservbyport($port, "tcp") || grep(/[:.]$port\s/, @nstat));
$delta = int(rand(32768));
}
return $port;
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by "Malte S. Stretz" <ms...@gmx.net>.
On Saturday 27 November 2004 03:28 CET Theo Van Dinter wrote:
> On Fri, Nov 26, 2004 at 05:34:00PM -0800, Justin Mason wrote:
> > wow guys -- overkill ;) I think both approaches are wrong.
>
> +1 to his mail in general. :)
+1 on any implementation :) The netstat solution was the first which came
to my mind when I ran into an acutally used port, everything more efficient
is of course cool. Maybe I'll find time to do it but I'm currently
knee-depth in some weird Delphi (sic) project for University.
Cheers,
Malte
--
[SGT] Simon G. Tatham: "How to Report Bugs Effectively"
<http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>
[ESR] Eric S. Raymond: "How To Ask Questions The Smart Way"
<http://www.catb.org/~esr/faqs/smart-questions.html>
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by Theo Van Dinter <fe...@kluge.net>.
On Fri, Nov 26, 2004 at 05:34:00PM -0800, Justin Mason wrote:
> wow guys -- overkill ;) I think both approaches are wrong.
+1 to his mail in general. :)
--
Randomly Generated Tagline:
Real theology is always rather shocking to people who already
think they know what they think. I'm still shocked myself. :-)
-- Larry Wall in <19...@wall.org>
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by "Malte S. Stretz" <ms...@gmx.net>.
On Saturday 27 November 2004 00:20 CET Sidney Markowitz wrote:
> Malte S. Stretz wrote:
> > oops :) But I'm glad you didn't notice my HOMEx debugging glitch :)
>
> I did, but I understood what it was for :-)
>
> I spoke too soon about it working. When I add a -w to the perl command
> it barfs in catpath, I think because it expects to be passed all three
> arguments, volume, dir, and file. I'll try adding a third argument of ''
> and see what it does. Or I could try reading the doc on catpath first :)
*grr*, you're right. I was pretty sure I said -w when I tried it... I
checked in a version of the code into ArchiveIterator, could you please fix
it there? Now that I think about it should it probably also check if those
other environment vars are defined.
Cheers,
Malte
--
[SGT] Simon G. Tatham: "How to Report Bugs Effectively"
<http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>
[ESR] Eric S. Raymond: "How To Ask Questions The Smart Way"
<http://www.catb.org/~esr/faqs/smart-questions.html>
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by Sidney Markowitz <si...@sidney.com>.
Malte S. Stretz wrote:
> oops :) But I'm glad you didn't notice my HOMEx debugging glitch :)
I did, but I understood what it was for :-)
I spoke too soon about it working. When I add a -w to the perl command
it barfs in catpath, I think because it expects to be passed all three
arguments, volume, dir, and file. I'll try adding a third argument of ''
and see what it does. Or I could try reading the doc on catpath first :)
-- sidney
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by "Malte S. Stretz" <ms...@gmx.net>.
On Friday 26 November 2004 22:07 CET Sidney Markowitz wrote:
> Malte S. Stretz wrote:
> > What do you think about the attached patch? Try it with
> > perl -Ilib -MMail::SpamAssassin::Util \
> > -le 'print Mail::SpamAssassin::Util::get_home()'
>
> It's HOMEPATH not HOMEDIR,
oops :) But I'm glad you didn't notice my HOMEx debugging glitch :)
> but with that change the Windows branch of it
> works fine as long as you use " instead of ' in the brain damaged
> Windows command line :-)
Hmmm... whever invented the cmd.exe shell syntax must have been on duope or
something. Which reminds me of my "Scripting Windows" article [1] I wanted
to write a second and a third part of...
Cheers,
Malte
[1] http://msquadrat.de/archive/04/02/24/01
--
[SGT] Simon G. Tatham: "How to Report Bugs Effectively"
<http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>
[ESR] Eric S. Raymond: "How To Ask Questions The Smart Way"
<http://www.catb.org/~esr/faqs/smart-questions.html>
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by Sidney Markowitz <si...@sidney.com>.
Malte S. Stretz wrote:
> What do you think about the attached patch? Try it with
> perl -Ilib -MMail::SpamAssassin::Util \
> -le 'print Mail::SpamAssassin::Util::get_home()'
It's HOMEPATH not HOMEDIR, but with that change the Windows branch of it
works fine as long as you use " instead of ' in the brain damaged
Windows command line :-)
-- sidney
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by "Malte S. Stretz" <ms...@gmx.net>.
On Friday 26 November 2004 21:42 CET Sidney Markowitz wrote:
> Malte S. Stretz wrote:
> > So maybe we should add a M::SA::Util::get_home() which first
> > tries $ENV{HOME}, then on Windows $ENV{HOMEDRIVE}\$ENV{HOMEDIR}, then
> > portable_getpwuid()[7], then... foo?
>
> portable_getpwuid() doesn't seem to do anything useful under Windows for
> this purpose and shouldn't be needed anyway. It just returns 'unknown'
> for the name, which works when you don't care about the actual user name.
Yeah, noticed that, too.
> The first two steps are fine, and probably enough, except that you would
> not have to add the '\' separator, it is already in HOMEDIR.
>
> Question: In ArchiveIterator.pm does everything work if that is what it
> uses for HOME or does anything have to be done to convert \ to / ?
What do you think about the attached patch? Try it with
perl -Ilib -MMail::SpamAssassin::Util \
-le 'print Mail::SpamAssassin::Util::get_home()'
Cheers,
Malte
--
[SGT] Simon G. Tatham: "How to Report Bugs Effectively"
<http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>
[ESR] Eric S. Raymond: "How To Ask Questions The Smart Way"
<http://www.catb.org/~esr/faqs/smart-questions.html>
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by Sidney Markowitz <si...@sidney.com>.
Malte S. Stretz wrote:
> So maybe we should add a M::SA::Util::get_home() which first
> tries $ENV{HOME}, then on Windows $ENV{HOMEDRIVE}\$ENV{HOMEDIR}, then
> portable_getpwuid()[7], then... foo?
portable_getpwuid() doesn't seem to do anything useful under Windows for
this purpose and shouldn't be needed anyway. It just returns 'unknown'
for the name, which works when you don't care about the actual user name.
The first two steps are fine, and probably enough, except that you would
not have to add the '\' separator, it is already in HOMEDIR.
Question: In ArchiveIterator.pm does everything work if that is what it
uses for HOME or does anything have to be done to convert \ to / ?
-- sidney
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by "Malte S. Stretz" <ms...@gmx.net>.
On Friday 26 November 2004 21:13 CET Theo Van Dinter wrote:
> On Fri, Nov 26, 2004 at 09:08:53PM +0100, Malte S. Stretz wrote:
> > Hm. The line is
> > $path =~ s!^~/!$ENV{'HOME'}/!;
> > so probably HOME is not set. The question is, what should this routine
> > do if HOME is not set; probably the value from (getpwuid($<))[7]
> > (before $< is changed). What does getpwuid() say on Windows?
>
> Doesn't matter really, we use M::SA::Util::portable_getpwuid() ... :)
Ah, cool :) So maybe we should add a M::SA::Util::get_home() which first
tries $ENV{HOME}, then on Windows $ENV{HOMEDRIVE}\$ENV{HOMEDIR}, then
portable_getpwuid()[7], then... foo?
Cheers,
Malte
--
[SGT] Simon G. Tatham: "How to Report Bugs Effectively"
<http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>
[ESR] Eric S. Raymond: "How To Ask Questions The Smart Way"
<http://www.catb.org/~esr/faqs/smart-questions.html>
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by Sidney Markowitz <si...@sidney.com>.
Malte S. Stretz wrote:
> What does getpwuid() say on Windows?
Not implemented :-)
You can't use getpwuid in Windows. The usual portable implementation
checks for running under Windows and uses $ENV{'HOMEDRIVE'} .
$ENV{'HOMEPATH'} if it is instead of $ENV{'HOME'}, being careful about
the former using '\' separators instead of '/'.
-- sidney
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by Theo Van Dinter <fe...@kluge.net>.
On Fri, Nov 26, 2004 at 09:08:53PM +0100, Malte S. Stretz wrote:
> Hm. The line is
> $path =~ s!^~/!$ENV{'HOME'}/!;
> so probably HOME is not set. The question is, what should this routine do
> if HOME is not set; probably the value from (getpwuid($<))[7] (before $< is
> changed). What does getpwuid() say on Windows?
Doesn't matter really, we use M::SA::Util::portable_getpwuid() ... :)
--
Randomly Generated Tagline:
"Now this really annoys me: all these people getting on the Internet and
saying Nostradamus predicted this. If Nostradamus were alive today his
name would be Miss Cleo and he'd be charging $2.99 a minute." - Jay Leno
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by "Malte S. Stretz" <ms...@gmx.net>.
On Friday 26 November 2004 20:30 CET Sidney Markowitz wrote:
> I just tried a quick build and make test in Windows XP to see what it
> would do, and
>
> 1. I could not reach the svn server from svn, although I could ping it.
> Is it down?
>
> 2. I got lots and lots of
>
> Use of uninitialized value in concatenation (.) or string at
> ..\lib/Mail/SpamAssassin/ArchiveIterator.pm line 1023.
Hm. The line is
$path =~ s!^~/!$ENV{'HOME'}/!;
so probably HOME is not set. The question is, what should this routine do
if HOME is not set; probably the value from (getpwuid($<))[7] (before $< is
changed). What does getpwuid() say on Windows?
> 3. I realized that I would not be able to test the use of netstat anyway
> because Windows does not run spamd. You can set environment variables to
> tell the spamc tests to assume that spamd is already running on some ip
> address and port, but that isn't relevant to this issue.
The routine is used in the initialization for $spamdport which is run for
every test. So you could add a print line in SATest.pm:63. But it should
be enough if that routine just doesn't print any errors :)
Cheers,
Malte
--
[SGT] Simon G. Tatham: "How to Report Bugs Effectively"
<http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>
[ESR] Eric S. Raymond: "How To Ask Questions The Smart Way"
<http://www.catb.org/~esr/faqs/smart-questions.html>
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by Sidney Markowitz <si...@sidney.com>.
The error message from ArchiveIterator.pm is because Windows does not
define $HOME environment variable by default. It has $HOMEDRIVE and
$HOMEPATH which together server the same purpose. The code in
ArchiveIterator.pm has to be changed to check for Windows, or else we
can document the need to set a $HOME. Do we use $HOME anywhere else?
I just found it because I used to have HOME defined in my XP environment
for other reasons.
-- sidney
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by "Malte S. Stretz" <ms...@gmx.net>.
On Friday 26 November 2004 20:41 CET Theo Van Dinter wrote:
> On Fri, Nov 26, 2004 at 01:39:33PM -0600, Michael Parker wrote:
> > > 1. I could not reach the svn server from svn, although I could ping
> > > it. Is it down?
> >
> > Yes, some sort of db problem, they are re-loading the database now.
>
> Watching SVN at Apache does not instill confidence in me about the
> reliability of said software.
Did this happen before? The error message did make me feel a bit weird,
especially because svn didn't tell if it was on my side or the server
(opening the page in my browser did assure me though that it was indeed the
server).
Cheers,
Malte
--
[SGT] Simon G. Tatham: "How to Report Bugs Effectively"
<http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>
[ESR] Eric S. Raymond: "How To Ask Questions The Smart Way"
<http://www.catb.org/~esr/faqs/smart-questions.html>
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by Michael Parker <pa...@pobox.com>.
On Fri, Nov 26, 2004 at 02:41:14PM -0500, Theo Van Dinter wrote:
> On Fri, Nov 26, 2004 at 01:39:33PM -0600, Michael Parker wrote:
> > > 1. I could not reach the svn server from svn, although I could ping it.
> > > Is it down?
> >
> > Yes, some sort of db problem, they are re-loading the database now.
>
> Watching SVN at Apache does not instill confidence in me about the reliability
> of said software.
It does have 100k+ revisions. Looks like some sort of odd DBD error
that they are gonna get Sleepycat to look into.
Oh yeah, and it's back up now.
Michael
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by Theo Van Dinter <fe...@kluge.net>.
On Fri, Nov 26, 2004 at 01:39:33PM -0600, Michael Parker wrote:
> > 1. I could not reach the svn server from svn, although I could ping it.
> > Is it down?
>
> Yes, some sort of db problem, they are re-loading the database now.
Watching SVN at Apache does not instill confidence in me about the reliability
of said software.
--
Randomly Generated Tagline:
Computer Artist died, GIF at Eleven.
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by Michael Parker <pa...@pobox.com>.
On Sat, Nov 27, 2004 at 08:30:30AM +1300, Sidney Markowitz wrote:
>
> 1. I could not reach the svn server from svn, although I could ping it.
> Is it down?
>
Yes, some sort of db problem, they are re-loading the database now.
Michael
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by Sidney Markowitz <si...@sidney.com>.
I just tried a quick build and make test in Windows XP to see what it
would do, and
1. I could not reach the svn server from svn, although I could ping it.
Is it down?
2. I got lots and lots of
Use of uninitialized value in concatenation (.) or string at
..\lib/Mail/SpamAssassin/ArchiveIterator.pm line 1023.
3. I realized that I would not be able to test the use of netstat anyway
because Windows does not run spamd. You can set environment variables to
tell the spamc tests to assume that spamd is already running on some ip
address and port, but that isn't relevant to this issue.
-- sidney
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by "Malte S. Stretz" <ms...@gmx.net>.
On Friday 26 November 2004 06:48 CET Daniel Quinlan wrote:
> mss@apache.org writes:
> > What's the probability that I run into an already used port with the
> > new probably_unused_spamd_port() code? Less than 1 per mill? Ask
> > Murphy...
>
> The only chance of a collision is if the port is listed in
> /etc/services.
If I understood that code right (which I hope), is the only safety which was
in there that the port is *not* in /etc/services.
> My system only has 3 TCP ports above 32768 listed. [...]
That means that in 32768 - 3 cases it was not checked for a collision. Now,
if it's 32768 or 32765, in my eyes it still pretty unrealistic that you hit
an already-used port twice. That's why I asked for Murphy...
Actually, I now know why it happened: I played around with a port knocking
daemon written in Java before. And that one was listening on ~1024 port
above 32768 and spamd tried to use a port out of that range.
> > The routine now tries to ask netstat if that port is already in use.
> > I tested the pattern on Linux, FreeBSD and Windows. If netstat can't
> > be run, no harm is done, the routine will just work as before. The
> > grep is pretty broad, it might also catch a remote port; then it just
> > tries the next random one. (Hey Murphy, it really can't hit a used
> > port ten times, can it?)
>
> I'm not a big fan of shell calls, but it looks (untested) like it'll
> work on Windows XP too.
I haven't actively tested it on Windows XP, but I had a look at the netstat
output there. That's why the first grep looks for spaces in front of the
"tcp" string (and it doesn't look for spaces afterwards because FreeBSD
reports the type as "tcp4"). Windows XP also has one column less than the
Unixes.
Even if it doesn't work on XP, it wouldn't matter. Then the @nstat array is
empty and the code just works as it did before.
Cheers,
Malte
--
[SGT] Simon G. Tatham: "How to Report Bugs Effectively"
<http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>
[ESR] Eric S. Raymond: "How To Ask Questions The Smart Way"
<http://www.catb.org/~esr/faqs/smart-questions.html>
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by Daniel Quinlan <qu...@pathname.com>.
mss@apache.org writes:
> What's the probability that I run into an already used port with the
> new probably_unused_spamd_port() code? Less than 1 per mill? Ask
> Murphy...
The only chance of a collision is if the port is listed in
/etc/services. My system only has 3 TCP ports above 32768 listed. So
if my math is right, that's a 0.0000003% chance of a collision between
two processes. The purely random code had a 0.1% chance of a collision
between two processes (running at the same time which could happen),
mostly because it only used 1000 ports. A 32768-port random version
would have a 0.003% chance of a collision.
> The routine now tries to ask netstat if that port is already in use.
> I tested the pattern on Linux, FreeBSD and Windows. If netstat can't
> be run, no harm is done, the routine will just work as before. The
> grep is pretty broad, it might also catch a remote port; then it just
> tries the next random one. (Hey Murphy, it really can't hit a used
> port ten times, can it?)
I'm not a big fan of shell calls, but it looks (untested) like it'll
work on Windows XP too.
Daniel
--
Daniel Quinlan
http://www.pathname.com/~quinlan/
Re: svn commit: r106600 - /spamassassin/trunk/t/SATest.pm
Posted by Daniel Quinlan <qu...@pathname.com>.
mss@apache.org writes:
> What's the probability that I run into an already used port with the
> new probably_unused_spamd_port() code? Less than 1 per mill? Ask
> Murphy...
The only chance of a collision is if the port is listed in
/etc/services. My system only has 3 TCP ports above 32768 listed. So
if my math is right, that's a 0.0000003% chance of a collision between
two processes. The purely random code had a 0.1% chance of a collision
between two processes (running at the same time which could happen),
mostly because it only used 1000 ports. A 32768-port random version
would have a 0.003% chance of a collision.
> The routine now tries to ask netstat if that port is already in use.
> I tested the pattern on Linux, FreeBSD and Windows. If netstat can't
> be run, no harm is done, the routine will just work as before. The
> grep is pretty broad, it might also catch a remote port; then it just
> tries the next random one. (Hey Murphy, it really can't hit a used
> port ten times, can it?)
I'm not a big fan of shell calls, but it looks (untested) like it'll
work on Windows XP too.
Daniel
--
Daniel Quinlan
http://www.pathname.com/~quinlan/