You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Josh Trutwin <jo...@trutwins.homeip.net> on 2004/06/18 23:18:31 UTC
Why does it take minutes to generate a report? (WAS: spamd fast but
spamc slow)
I'm trying to fix my problems with my SA 2.63 installation (using Debian Stable, perl 5.6.1, Qmail, Qmail-scanner 1.22, clamav 0.73)
I have discovered that if I run:
spamc -f -c -u "josh@netbits.us" -U /var/spool/spamassassin/spamd.sock < sample-spam.txt
that the spamc process returns in about 2-3 seconds, as expected.
If I take out the -c to return the full message and report, it takes minutes, sometimes as long as 4 minutes! WFT?!?
Spamd is started with:
/usr/bin/spamd -d -a -q -x -H /etc/razor --socketpath=/var/spool/spamassassin/spamd.sock -m 10 -u spamd
I've tried it with -L instead of -a and the results are the same. I re-ran razor-admin -discover incase it was a timeout problem on a dead razor server. No change.
The main thing is that spamd does it's job quickly, but spamc sits and sits without the -c option. What could be causing this? I guess I can take out the -c option, but my users are used to the way it works now with the full report.
Thanks,
Josh
Re: Why does it take minutes to generate a report? (WAS: spamd fast
but spamc slow)
Posted by Josh Trutwin <jo...@trutwins.homeip.net>.
On Sat, 19 Jun 2004 13:28:03 -0500
Josh Trutwin <jo...@trutwins.homeip.net> wrote:
> Another test (run as spamd user):
>
> > date ; spamassassin -t -x -L -D < sample-spam.txt ; date
>
> Sat Jun 19 13:15:35 CDT 2004
>
> debug: Score set 0 chosen.
<snip>
>
> Sat Jun 19 13:16:50 CDT 2004
>
> Over a minute. Is there any way to tell what is causing this
> behavior?
Well, with little help but my own :( I figured out what was going on. I had a bad rule in my SARE ruleset, I think it was one of the blacklist rules. I have no idea why, but removing the rules, re-running rules_du_jour and re-linting everything caused the problem to go away, now I'm back to 1-3 second full-report peak scanning vs the 10 minute teeth-grinding madness. I also found out that I can use sa-blacklist as a qmail badmailfrom instead, rejecting at the SMTP level is ALWAYS a good thing. :)
Hopefully some other unfortunate soul will find this usefull.
Josh
Re: Why does it take minutes to generate a report? (WAS: spamd fast
but spamc slow)
Posted by Josh Trutwin <jo...@trutwins.homeip.net>.
Another test (run as spamd user):
> date ; spamassassin -t -x -L -D < sample-spam.txt ; date
Sat Jun 19 13:15:35 CDT 2004
debug: Score set 0 chosen.
debug: running in taint mode? yes
debug: Running in taint mode, removing unsafe env vars, and resetting PATH
debug: PATH included '/usr/local/bin', keeping.
debug: PATH included '/usr/bin', keeping.
debug: PATH included '/bin', keeping.
debug: PATH included '/usr/bin/X11', keeping.
debug: PATH included '/usr/games', keeping.
debug: PATH included '/usr/local/apache/bin', keeping.
debug: PATH included '/usr/local/mysql/bin', keeping.
debug: PATH included '/usr/local/pgsql/bin', keeping.
debug: PATH included '/usr/local/java/bin', keeping.
debug: PATH included '/usr/local/bglibs/bin', keeping.
debug: Final PATH set to: /usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games:/usr/local/apache/bin:/usr/local/mysql/bin:/usr/local/pgsql/bin:/usr/local/java/bin:/usr/local/bglibs/bin
debug: using "/usr/local/share/spamassassin" for default rules dir
debug: using "/etc/mail/spamassassin" for site rules dir
debug: using "/var/spool/spamassassin//.spamassassin/user_prefs" for user prefs file
debug: bayes: 23455 tie-ing to DB file R/O /var/spool/spamassassin/bayes_toks
debug: bayes: 23455 tie-ing to DB file R/O /var/spool/spamassassin/bayes_seen
debug: bayes: found bayes db version 2
^^^^^^^^^^^^
Is that ok?
debug: Score set 2 chosen.
debug: Initialising learner
debug: running header regexp tests; score so far=0
debug: running body-text per-line regexp tests; score so far=1.391
debug: bayes corpus size: nspam = 67217, nham = 5160
debug: uri tests: Done uriRE
debug: tokenize: header tokens for *M = " <snip> 1010101 example net "
debug: tokenize: header tokens for *F = "U*sender D*example.net D*net"
debug: tokenize: header tokens for To = "U*recipient D*example.net D*net"
debug: tokenize: header tokens for Precedence = "junk"
debug: tokenize: header tokens for MIME-Version = ""
debug: tokenize: header tokens for *c = "/plain; charset=us-ascii"
debug: tokenize: header tokens for Content-Transfer-Encoding = "7bit"
debug: bayes token 'Generic' => 0.999912183995438
debug: bayes token 'string' => 0.999129943502825
debug: bayes token 'detecting' => 0.997298245614035
debug: bayes token 'filter' => 0.993492957746479
debug: bayes token 'incoming' => 0.992426229508197
debug: bayes token 'Bulk' => 0.985096774193548
debug: bayes token 'HTo:D*net' => 0.0362795995765551
debug: bayes token 'H*c:plain' => 0.0672461089762293
debug: bayes token 'spaces' => 0.129430663357985
debug: bayes token 'breaks' => 0.863956241482732
debug: bayes: score = 0.843246259112419
debug: bayes: 23455 untie-ing
debug: bayes: 23455 untie-ing db_toks
debug: bayes: 23455 untie-ing db_seen
debug: running raw-body-text per-line regexp tests; score so far=1.391
debug: running uri tests; score so far=1.391
debug: uri tests: Done uriRE
debug: running full-text regexp tests; score so far=1.391
debug: all '*From' addrs: sender@example.net
debug: all '*To' addrs: recipient@example.net
debug: running meta tests; score so far=1.391
debug: auto-learn? ham=0.1, spam=12, body-hits=0, head-hits=1.391
debug: auto-learn: currently using scoreset 2. recomputing score based on scoreset 0.
debug: Score set 0 chosen.
debug: auto-learn: original score: 2.491, recomputed score: 2.491
debug: Score set 2 chosen.
debug: auto-learn? no: inside auto-learn thresholds
debug: is spam? score=1004.933 required=5 tests=BAYES_80,FVGT_m_MULTI_ODD2,<snip>,L_T_COMBINED
Received: from localhost by fastconcepts
with SpamAssassin (2.63 2004-01-11);
Sat, 19 Jun 2004 13:15:39 -0500
From: Sender <se...@example.net>
To: Recipient <re...@example.net>
Subject: Test spam mail (<snip>)
Date: Wed, 23 Jul 2003 23:30:00 +0200
Message-Id: <<s...@example.net>
X-Spam-Flag: YES
X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on fastconcepts
X-Spam-Status: Yes, hits=1004.9 required=5.0 tests=BAYES_80,FVGT_m_MULTI_ODD2,
<snip>,L_T_COMBINED autolearn=no version=2.63
X-Spam-Level: **************************************************
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----------=_40D482CB.5D95D814"
This is a multi-part message in MIME format.
------------=_40D482CB.5D95D814
Content-Type: text/plain
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
Spam detection software, running on the system <snip>
Content preview: <snip>
Content analysis details: (1004.9 points, 5.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
1.4 L_T_COMBINED Addressed to "Undisclosed Recipients" or equivalent
<snip>
2.4 BAYES_80 BODY: Bayesian spam probability is 80 to 90%
[score: 0.8432]
1.1 FVGT_m_MULTI_ODD2 Contains multiple odd letter combinations
------------=_40D482CB.5D95D814
Content-Type: message/rfc822; x-spam-type=original
Content-Description: original message before SpamAssassin
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
Subject: Test spam mail (<snip>)
Message-ID: <<s...@example.net>
Date: Wed, 23 Jul 2003 23:30:00 +0200
From: Sender <se...@example.net>
To: Recipient <re...@example.net>
Precedence: junk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
<snip>
------------=_40D482CB.5D95D814--
Spam detection software, running on the system <snip>
Content preview: This is the <snip>
Content analysis details: (1004.9 points, 5.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
1.4 L_T_COMBINED Addressed to "Undisclosed Recipients" or equivalent
<snip>
2.4 BAYES_80 BODY: Bayesian spam probability is 80 to 90%
[score: 0.8432]
1.1 FVGT_m_MULTI_ODD2 Contains multiple odd letter combinations
After this is printed, spamassassin sits for about a minute, then the date gets printed (from the original command line)
Sat Jun 19 13:16:50 CDT 2004
Over a minute. Is there any way to tell what is causing this behavior?
Thanks,
Josh