You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@bugzilla.spamassassin.org on 2004/12/04 00:01:47 UTC
[Bug 4016] New: excessive use of fds
http://bugzilla.spamassassin.org/show_bug.cgi?id=4016
Summary: excessive use of fds
Product: Spamassassin
Version: 3.0.0
Platform: All
OS/Version: other
Status: NEW
Severity: major
Priority: P2
Component: spamassassin
AssignedTo: dev@spamassassin.apache.org
ReportedBy: pobox@rgrs.com
Spamassassin v3.0.0 appears to be using an excessive amount
of open file descriptors for dns use. This is likely a bug.
The exact amount depends on the message being processed.
For example, the particular spam I've been testing with
below needs at least 97 open fds to successfully complete.
Many ISPs set the process limit to 64.
You can tell if you've hit this bug by the following error:
razor2 check skipped: No such file or directory
IO::Socket::INET: Bad protocol 'udp' ...propagated at
/pkg/Mail-SpamAssassin-3.000000-580/lib/site_perl/5.8.0/Mail/SpamAssassin/Dns.pm
line 447.
Here's a patch:
*** /usr/local/test-sa/lib/site_perl/5.8.0/Mail/SpamAssassin/Util.pm Mon Sep 13
19:34:05 2004
--- Mail/SpamAssassin/Util.pm Fri Dec 3 15:31:59 2004
***************
*** 724,766 ****
###########################################################################
- # thanks to http://www2.picante.com:81/~gtaylor/autobuse/ for this
- # code.
sub secure_tmpfile {
! my $tmpdir = Mail::SpamAssassin::Util::untaint_file_path(
! File::Spec->tmpdir()
! );
! if (!$tmpdir) {
! die "Cannot find a temporary directory! set TMP or TMPDIR in env";
! }
! my ($reportfile,$tmpfile);
! my $umask = umask 077;
! do {
# we do not rely on the obscurity of this name for security...
# we use a average-quality PRG since this is all we need
! my $suffix = join ('',
! (0..9, 'A'..'Z','a'..'z')[rand 62,
! rand 62,
! rand 62,
! rand 62,
! rand 62,
! rand 62]);
! $reportfile = File::Spec->catfile(
! $tmpdir,
! join ('.',
! "spamassassin",
! $$,
! $suffix,
! "tmp",
! )
! );
# ...rather, we require O_EXCL|O_CREAT to guarantee us proper
# ownership of our file; read the open(2) man page.
! } while (! sysopen ($tmpfile, $reportfile, O_RDWR|O_CREAT|O_EXCL, 0600));
! umask $umask;
! return ($reportfile, $tmpfile);
}
###########################################################################
--- 724,766 ----
###########################################################################
sub secure_tmpfile {
! my ($reportfile, $suffix, $tmpdir, $tmpfile, $umask);
! my $count = 0;
!
! $tmpdir = Mail::SpamAssassin::Util::untaint_file_path(File::Spec->tmpdir)
! or die "Cannot find a temporary directory! set TMP or TMPDIR in env";
!
! $umask = umask 077;
! while (1) {
# we do not rely on the obscurity of this name for security...
# we use a average-quality PRG since this is all we need
! $suffix = join ('', (0..9,'A'..'Z','a'..'z')[rand 62,
! rand 62,
! rand 62,
! rand 62,
! rand 62,
! rand 62]);
! $reportfile = File::Spec->catfile($tmpdir, join('.',
! "spamassassin",
! $$,
! $suffix,
! "tmp"));
! die "Problem creating temporary file" if ++$count > 100_000;
!
! next if -e $reportfile;
!
# ...rather, we require O_EXCL|O_CREAT to guarantee us proper
# ownership of our file; read the open(2) man page.
! sysopen $tmpfile, $reportfile, O_RDWR|O_CREAT|O_EXCL, 0600
! or die "Cannot create temporary file \"$reportfile\": $!";
!
! last;
! }
! umask $umask;
! return($reportfile, $tmpfile);
}
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4016] excessive use of fds
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4016
------- Additional Comments From pobox@rgrs.com 2004-12-03 15:13 -------
Created an attachment (id=2545)
--> (http://bugzilla.spamassassin.org/attachment.cgi?id=2545&action=view)
the patch in desc is for bug #4017 - here's the testcase
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4016] excessive use of fds
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4016
spamassassin-contrib@msquadrat.de changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |DUPLICATE
------- Additional Comments From spamassassin-contrib@msquadrat.de 2004-12-03 15:48 -------
*** This bug has been marked as a duplicate of 4017 ***
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
Re: [Bug 4016] New: excessive use of fds
Posted by Tony Finch <do...@dotat.at>.
I've been doing some DNS-intensive work with ADNS recently, and I was
reminded how fast it is and how easy it is to run bulk jobs with over
10,000 concurrent DNS queries. You only need two sockets! Maybe I should
beat Net::DNS to death with the clue bat.
http://www.livejournal.com/users/fanf/
http://www.chiark.greenend.org.uk/~ian/adns/
Tony.
--
f.a.n.finch <do...@dotat.at> http://dotat.at/
MALIN HEBRIDES: NORTHEAST 4 OR 5 INCREASING 6. RAIN LATER. GOOD BECOMING
MODERATE.
[Bug 4016] excessive use of fds
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4016
------- Additional Comments From jm@jmason.org 2004-12-03 16:54 -------
I believe there's an FAQ about it.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4016] excessive use of fds
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4016
pobox@rgrs.com changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |REOPENED
Resolution|DUPLICATE |
------- Additional Comments From pobox@rgrs.com 2004-12-03 16:01 -------
Bugs 4016 and 4017 really are separate bugs. True, the
patch given in the description of 4016 does belong with bug
4017. However, the issue of excessive use of fds is
completely separate from the infinite loop bug listed in
#4017.
Someone needs to either diagnose and solve this, or describe
in the INSTALL notes how many open fds are required for
spamassassin to work correctly and also note that the
typical limit of 64 fds doesn't work.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.