You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@bugzilla.spamassassin.org on 2005/08/05 21:29:17 UTC
[Bug 4518] New: Helpers like dccproc and pyzor left running even after SA times out on them
http://bugzilla.spamassassin.org/show_bug.cgi?id=4518
Summary: Helpers like dccproc and pyzor left running even after
SA times out on them
Product: Spamassassin
Version: SVN Trunk (Latest Devel Version)
Platform: All
OS/Version: All
Status: NEW
Severity: normal
Priority: P3
Component: Libraries
AssignedTo: dev@spamassassin.apache.org
ReportedBy: Mark.Martinec@ijs.si
Watching logs with Perl warnings turned on, the following caught
my eye (explanation from the perldiag 5.8.7 man page):
Warning: unable to close filehandle %s properly
(S) The implicit close() done by an open() got an error
indication on the close().
where %s was either DCC or PYZOR (I'm using dccproc and pyzor helpers).
It turned out that 5 second timeouts on these unresponsive helper
applications left the *DCC or *PYZOR file descriptors unclosed
(the 'close DCC' in eval was skipped because of alarm signal),
and the spawned helpers were left running.
Eventually when the next mail came around for checking within the
same content filtering process (this was with amavisd-new, but I don't
think it is specific to that environment), the SA was called again,
and Util::helper_app_pipe_open_unix tried to re-open the (still open)
*DCC file descriptor:
my $pid = open ($fh, '-|');
which implicitly invoked close on the still open file handle,
this resulting in a Perl warning, as the previous pipe had a non-zero
exit status pending.
I'm not sure how serious this it. It may just mean that CPU cycles
are wasted on running slow helper programs beyond the point where
SA has already given up on them; or there may be some interaction
between the successive invocations of a helper program, perhaps
through its temporary files. It seems it has to do something with
seeing 'dcc: failed to read header'.
In an attempt to track down the situation and perhaps rectify it,
I came up with the fallowing quick-and-dirty change to Plugin/DCC.pm
(and equivalent for Plugin/Pyzor.pm):
--- DCC.pm~ Sun Jul 31 07:08:27 2005
+++ DCC.pm Fri Aug 5 20:40:29 2005
@@ -459,2 +459,3 @@
+ my $pid;
eval {
@@ -474,3 +475,3 @@
- my $pid = Mail::SpamAssassin::Util::helper_app_pipe_open(*DCC,
+ $pid = Mail::SpamAssassin::Util::helper_app_pipe_open(*DCC,
$tmpf, 1, $path, "-H", split(' ', $opts));
@@ -510,2 +511,11 @@
my $err = $@;
+
+ if (defined(fileno(*DCC))) { # still open
+ if ($pid) {
+ if (kill('TERM',$pid)) { dbg("dcc: killed stale helper [$pid]") }
+ else { dbg("dcc: killing helper application [$pid] failed: $!") }
+ }
+ close DCC or dbg(sprintf("dcc: pipe closed: %s exit=0x%04x", $!,$?));
+ }
which explicitly closes the *DCC (and *PYZOR) file handle if
left open, and explicitly terminates the abandoned helper process.
It seems to do good: no more "Warning: unable to close filehandle %s properly",
and no more 'failed to read header'. The log now reports killings and
the 5 second timeouts when dccproc and pyzor are unresponsive.
As a possible improvement a flag may be used instead of testing
the fileno(*DCC) whether the file is still open. Even better would probably
be to use IO::File -based file descriptor instead of a static ones like *DCC,
which has a benefit of doing an implicit close in case when references
to it come out of scope, e.g. in case of unhandled error condition.
The Mail::SpamAssassin::Util::helper_app_pipe_open could create and
return a new such handle, besides returning the $pid. Just an idea.
Mark
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4518] [review] Helpers like dccproc and pyzor left running even after SA times out on them
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4518
jm@jmason.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Status Whiteboard| |needs 2 votes
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4518] [review] Helpers like dccproc and pyzor left running even after SA times out on them
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4518
jm@jmason.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |FIXED
------- Additional Comments From jm@jmason.org 2005-08-19 20:03 -------
applied; r23399[01]
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4518] Helpers like dccproc and pyzor left running even after SA times out on them
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4518
------- Additional Comments From Mark.Martinec@ijs.si 2005-08-19 08:08 -------
Created an attachment (id=3089)
--> (http://bugzilla.spamassassin.org/attachment.cgi?id=3089&action=view)
updated patch to apply cleanly
The attached patch is essentially the same as the original
in-line one, but applies cleanly to the current code.
Apart from the originally described symptoms, it is also
a cure to the zombies problem, as discussed on the 'users'
mailing list these days (reported by Jim Knuth).
It is a minimalistic patch. A cleaner solution may be desired for 3.2.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4518] [review] Helpers like dccproc and pyzor left running even after SA times out on them
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4518
duncf@debian.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Status Whiteboard|needs 2 votes |ready to be committed
------- Additional Comments From duncf@debian.org 2005-08-19 16:14 -------
+1
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4518] [review] Helpers like dccproc and pyzor left running even after SA times out on them
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4518
duncf@debian.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Status Whiteboard|ready to be committed |1 more vote
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4518] [review] Helpers like dccproc and pyzor left running even after SA times out on them
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4518
sidney@sidney.com changed:
What |Removed |Added
----------------------------------------------------------------------------
Status Whiteboard|1 more vote |
------- Additional Comments From sidney@sidney.com 2005-08-19 16:35 -------
+1
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4518] [review] Helpers like dccproc and pyzor left running even after SA times out on them
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4518
------- Additional Comments From automasschecker@jmason.org 2005-08-19 16:11 -------
Subject: Re: [review] Helpers like dccproc and pyzor left running even after SA times out on them
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
> Does this need to be fixed for razor too?
no, it doesn't fork a process.
- --j.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Exmh CVS
iD8DBQFDBmZCMJF5cimLx9ARAt5hAKC8BLxfF/fGLcZgH6QwOqEfBmQxMACgruYM
NFS/kCtvtPoiNwkoPyL94Uo=
=Hnbh
-----END PGP SIGNATURE-----
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4518] [review] Helpers like dccproc and pyzor left running even after SA times out on them
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4518
jm@jmason.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Summary|Helpers like dccproc and |[review] Helpers like
|pyzor left running even |dccproc and pyzor left
|after SA times out on them |running even after SA times
| |out on them
Target Milestone|Undefined |3.1.0
------- Additional Comments From jm@jmason.org 2005-08-19 09:32 -------
+1 -- looks good to me
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 4518] [review] Helpers like dccproc and pyzor left running even after SA times out on them
Posted by bu...@bugzilla.spamassassin.org.
http://bugzilla.spamassassin.org/show_bug.cgi?id=4518
------- Additional Comments From duncf@debian.org 2005-08-19 16:03 -------
Does this need to be fixed for razor too?
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.