You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ruleqa@spamassassin.apache.org by Henrik K <he...@hege.li> on 2018/11/06 18:31:07 UTC

Please use --all mass-check option

Sigh...  just noticed that mass-check / automasscheck-minimal.cf skips
messages over 256 KB by default.  As trunk has already implemented
body_part_scan_size, there should be no reason not to scan large messages,
unless you are really low on memory.

For me that meant 1k messages out of 15k skipped!  Certainly could shift the
results slightly.

Note the comment I've added to automasscheck-minimal.cf.dist:

# Note the --all option which scans all messages regardless of size.
#   Without --all, default is to skip messages over 256 KB.
run_all_masschecks() {
  ### sample: single corpus ###
  run_masscheck single-corpus --all \
          --after=-174182400 ham:dir:/path/to/Maildir/.Ham/ \
          --after=-4838400 spam:dir:/path/to/Maildir/.Spam/

Try to use that..

Btw SA does eat a LOT of memory parsing large messages..  still have to work
on this..  I'll also add --max-size option to mass-check.  But perhaps clean
your corpus of huge messages.  :-)

msgsize rss
10k 130MB
1MB 150MB
10MB 400MB
30MB 1GB

Re: Please use --all mass-check option

Posted by Henrik K <he...@hege.li>.
Sigh nr.2..  old automasscheck-minimal.sh didn't even use trusted_networks
settings at all since user_prefs was wrong place for them.

Please upgrade to newest and check anyway that your *_NETWORKS are defined
properly.

https://svn.apache.org/repos/asf/spamassassin/trunk/masses/contrib/automasscheck-minimal/automasscheck-minimal.sh
https://svn.apache.org/repos/asf/spamassassin/trunk/masses/contrib/automasscheck-minimal/automasscheck-minimal.cf.dist

(Also added back bayes_auto_learn 0...  in case you wonder why today some
bayes databases appeared..)

On Tue, Nov 06, 2018 at 08:31:07PM +0200, Henrik K wrote:
> 
> Sigh...  just noticed that mass-check / automasscheck-minimal.cf skips
> messages over 256 KB by default.  As trunk has already implemented
> body_part_scan_size, there should be no reason not to scan large messages,
> unless you are really low on memory.
> 
> For me that meant 1k messages out of 15k skipped!  Certainly could shift the
> results slightly.
> 
> Note the comment I've added to automasscheck-minimal.cf.dist:
> 
> # Note the --all option which scans all messages regardless of size.
> #   Without --all, default is to skip messages over 256 KB.
> run_all_masschecks() {
>   ### sample: single corpus ###
>   run_masscheck single-corpus --all \
>           --after=-174182400 ham:dir:/path/to/Maildir/.Ham/ \
>           --after=-4838400 spam:dir:/path/to/Maildir/.Spam/
> 
> Try to use that..
> 
> Btw SA does eat a LOT of memory parsing large messages..  still have to work
> on this..  I'll also add --max-size option to mass-check.  But perhaps clean
> your corpus of huge messages.  :-)
> 
> msgsize rss
> 10k 130MB
> 1MB 150MB
> 10MB 400MB
> 30MB 1GB