enabling pyzor or dcc hangs spamd

[Walter Haidinger <wa...@gmx.at>]

Hi!

Whenever I enable either pyzor or dcc (or both), spamd hangs and spamc 
doesn't return until the spamd process is killed. While running, it tries 
to consume all available CPU time.

In short (more details below), here are the debug logs:
  debug: Pyzor is available: /usr/bin/pyzor
  debug: entering helper-app run mode
  logmsg: server hit by SIGCHLD
  server hit by SIGCHLD

That is, the helper-app never returns. Only if the forked spamd child is 
manually killed.

Without pyzor/dcc, or when fed through spamassassin (with dcc and pyzor), 
there are no problems.

I had this issue already in the past with Spamassassin 2.64. Back then I 
thought the problem would resolve with v3. Apparently, as with v3.0.2, it 
still persists! :-)

Is it possible to somehow debug the helper-app run mode?

But now on to the details:

Versions used:
OS: SuSE 9.1 with vanilla 2.6.10 kernel
Spamassassin: 3.0.2-1.1 SuSE binary RPM
razor: 	      2.61-3 SuSE binary RPM
pyzor:        0.4.0 compiled from source tarball
dcc:          1.2.68 compiled from source tarball

Once started, I run spamd with the following options:
/usr/sbin/spamd -D --daemonize --max-children=5 
   --pidfile=/var/run/spamd.pid --paranoid --create-prefs --user-config 
   --siteconfigpath=/etc/spamassassin 
   --helper-home-dir=/var/lib/home/spamd 
   --socketpath=/var/lib/home/spamd/spamd.socket

I've put some debug-logfiles to:
http://members.kabsi.at/wh/ml/spamassassin/20050127/

You'll find the following files there:
* testmsg
  the test message used (yes, that IS the mailing-list subscription 
  confirmation request)
* spamassassin.log
  _working_ spamassassin run with pyzor and dcc
* spamd-startup.log
  debugs log of spamd startup (no problem there, I think)
* spamd-dcc.log
  debug logs with dcc enabled (hangs)
* spamd-pyzor.log
  debug logs with pyzor enabled (hangs)
* spamd-working.log
  a working spamd run with both dcc and pyzor disabled
* all_logs.tar.gz
  a tarball of all files above

spamc was always invoked with:
'cat testmsg | spamc -U /var/lib/home/spamd/spamd.socket'

If there is anything missing, please reply and just ask for it. 
Thanks!

Thanks again if you made it until here, I appreciate that!

Regards, Walter

Re: enabling pyzor or dcc hangs spamd

[Walter Haidinger <wa...@gmx.at>]

On Thu, 27 Jan 2005, Martin Karol Zuziak wrote:

> Have you tried running spamc/spamd in regular tcp/ip mode instead of
> through unix sockets?

No, not yet because using domain sockets should perform better.
I'll try this though, good suggestion!

> Is it spamd that hogs the cpu? Does it hang before executing pyzor,
> while pyzor is executing or after pyzor terminates? You might find
> helpful information by attaching (with the -p switch) strace or ltrace
> to the process hogging the cpu. The hanging process is clearly doing
> something, but what?

Yes, it's definitely spamd who is hogging the cpu. As far as I can tell, 
it starts hogging right after (or when) the helper app (dcc or pyzor) is 
launched. And yes, attaching strace is a another good idea! Why didn't I 
think about that in the first place? ;-)

I'll test and reply asap (that is tomorrow, I hope).

Thanks, 
Walter

Re: enabling pyzor or dcc hangs spamd

[Martin Karol Zuziak <zu...@math.ku.dk>]

On Thu, Jan 27, 2005 at 04:41:56PM +0100, Walter Haidinger wrote:
> Hi!
> 
> Whenever I enable either pyzor or dcc (or both), spamd hangs and spamc 
> doesn't return until the spamd process is killed. While running, it tries 
> to consume all available CPU time.
> 
> In short (more details below), here are the debug logs:
>   debug: Pyzor is available: /usr/bin/pyzor
>   debug: entering helper-app run mode
>   logmsg: server hit by SIGCHLD
>   server hit by SIGCHLD
> 
> That is, the helper-app never returns. Only if the forked spamd child is 
> manually killed.
> 
> Without pyzor/dcc, or when fed through spamassassin (with dcc and pyzor), 
> there are no problems.

<snip>

> spamc was always invoked with:
> 'cat testmsg | spamc -U /var/lib/home/spamd/spamd.socket'

I am not that familiar with SA but here are some suggestions:

Have you tried running spamc/spamd in regular tcp/ip mode instead of
through unix sockets?

Is it spamd that hogs the cpu? Does it hang before executing pyzor,
while pyzor is executing or after pyzor terminates? You might find
helpful information by attaching (with the -p switch) strace or ltrace
to the process hogging the cpu. The hanging process is clearly doing
something, but what?

Good luck,

Martin Zuziak <zu...@math.ku.dk>