You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by aladdin Sorry about the generic <al...@csunv.com> on 2008/09/14 01:44:27 UTC
Spamassassin Letting a Lot of Spams Through
Sorry about the generic subject, but it is the only thing this newbie knows to
describe the symptom.
Platform: Debian (Etch?)
Latest Spamassassin in apt (version 3.1.7-deb)
Invocation comes from KMail, via spamc (presumably) to the spamd daemon- set
up using KMail Wizard, and manually checked
Spamassassin doesn't seem to be catching much spam at all. I've run thousands
of spams through sa-learn, and hundreds of hams (needless to say, I get the
ratio of thousands of spams to tens of hams). I can't see where it's even
using the Bayes filter.
Here is the config file:
############################################################################
# This is the right place to customize your installation of SpamAssassin.
#
# See 'perldoc Mail::SpamAssassin::Conf' for details of what can be
# tweaked.
#
# Only a small subset of options are listed below
#
###########################################################################
rewrite_header Subject *****SPAM*****
required_score 5.0
use_bayes 1
bayes_auto_learn 1
#############################################################################
And here's the german portion of an example header from a processed email that
should have been spam but wasn't:
############################################################################
X-Spam-Checker-Version: SpamAssassin 3.1.7-deb (2006-10-05) on
anw-dev.cfl.rr.com
X-Spam-Level:
X-Spam-Status: No, score=0.9 required=5.0 tests=SUBJ_HAS_UNIQ_ID autolearn=no
version=3.1.7-deb
X-Virus-Flag: no
Return-path: <ma...@ems.com>
<snip>
###########################################################################
It looks like, to this unwashed newbie, that: a) it's not autolearning
(perhaps it doesn't on real emails?) and b) even though this email is full of
references like "boosting your sexual power" and "high quality medications",
and even comes from address "admin@viagra.com", you can see it is still
getting a low spam score.
TIA
--
Thanks and regards,
Allen Williams
Office: +1.321.309.7931
Mobile: +1.321.258.1272
Re: Spamassassin Letting a Lot of Spams Through
Posted by aladdin <al...@csunv.com>.
On Sunday 14 September 2008 10:06, aladdin wrote:
> On Sunday 14 September 2008 05:07, you wrote:
> > On Sun, 2008-09-14 at 01:05 -0400, aladdin wrote:
> > > > So, evidently, it can't find my bayes database. So, since I want to
> > > > use a system-wide database, where is it (/usr/share/spamassassin?,
> > > > which has a lot of likely looking files in it), and how do I tell
> > > > spamd to use it?
> >
> > By default it is in the .spamassassin directory of the user SA runs in.
> >
> > Using /usr/share/spamassin sounds like a bad idea to me: you're
> > attempting to mix site-specific data with system files.
> >
> >
> > Martin
>
> Hmmm! Oddly enough, that's where apt (the Debian package manager) put
> them. So, I guess that leads to two more areas of questions:
>
> 1. Is there no precedent for stopping spam using system-wide files? I am
> almost the sole user of this machine and would like to do this, if it's
> possibe. Why would apt put them there otherwise?
>
> 2. If question one leads to user-specific files & directories, do I just
> take the contents of /usr/share/spamassassin and copy it into
> ~/.spamassassin? The contents of /usr/share/spamassassin are:
> ###########################################################
> total 676
> drwxr-xr-x 2 root root 4096 2008-09-07 19:24 ./
> drwxr-xr-x 256 root root 12288 2008-09-07 19:24 ../
> -rw-r--r-- 1 root root 5681 2007-02-15 00:28 10_misc.cf
<snip>
> -rw-r--r-- 1 root root 18944 2007-02-15 00:28 triplets.txt
> -rw-r--r-- 1 root root 1843 2007-02-15 00:28 user_prefs.template
> ####################################################################
> Are these the files to be copied to ~/.spamassassin?
As it turns out, I do have a ~/.spamassassin directory. It's current contents
are:
#################################################################
-rw------- 1 anw anw 1306624 2008-09-14 03:38 auto-whitelist
-rw------- 1 anw anw 88190 2008-07-28 16:52 bayes_journal
-rw------- 1 anw anw 684032 2008-07-28 16:52 bayes_seen
-rw------- 1 anw anw 5283840 2008-07-28 16:52 bayes_toks
-rw-r--r-- 1 anw anw 1487 2008-07-28 16:52 user_prefs
#################################################################
Should I just copy the above into it and change the owner/group, and that's
how spamassassin is supposed to work?
--
Thanks and regards,
anw
Re: Spamassassin Letting a Lot of Spams Through
Posted by aladdin <al...@csunv.com>.
On Sunday 14 September 2008 05:07, you wrote:
> On Sun, 2008-09-14 at 01:05 -0400, aladdin wrote:
> > > So, evidently, it can't find my bayes database. So, since I want to
> > > use a system-wide database, where is it (/usr/share/spamassassin?,
> > > which has a lot of likely looking files in it), and how do I tell spamd
> > > to use it?
>
> By default it is in the .spamassassin directory of the user SA runs in.
>
> Using /usr/share/spamassin sounds like a bad idea to me: you're
> attempting to mix site-specific data with system files.
>
>
> Martin
Hmmm! Oddly enough, that's where apt (the Debian package manager) put them.
So, I guess that leads to two more areas of questions:
1. Is there no precedent for stopping spam using system-wide files? I am
almost the sole user of this machine and would like to do this, if it's
possibe. Why would apt put them there otherwise?
2. If question one leads to user-specific files & directories, do I just take
the contents of /usr/share/spamassassin and copy it into ~/.spamassassin?
The contents of /usr/share/spamassassin are:
###########################################################
total 676
drwxr-xr-x 2 root root 4096 2008-09-07 19:24 ./
drwxr-xr-x 256 root root 12288 2008-09-07 19:24 ../
-rw-r--r-- 1 root root 5681 2007-02-15 00:28 10_misc.cf
-rw-r--r-- 1 root root 8327 2007-02-15 00:28 20_advance_fee.cf
-rw-r--r-- 1 root root 1791 2007-02-15 00:28 20_anti_ratware.cf
-rw-r--r-- 1 root root 7077 2007-02-15 00:28 20_body_tests.cf
-rw-r--r-- 1 root root 1749 2007-02-15 00:28 20_compensate.cf
-rw-r--r-- 1 root root 14505 2007-02-15 00:28 20_dnsbl_tests.cf
-rw-r--r-- 1 root root 15854 2007-02-15 00:28 20_drugs.cf
-rw-r--r-- 1 root root 11672 2007-02-15 00:28 20_fake_helo_tests.cf
-rw-r--r-- 1 root root 33045 2007-02-15 00:28 20_head_tests.cf
-rw-r--r-- 1 root root 17485 2007-02-15 00:28 20_html_tests.cf
-rw-r--r-- 1 root root 3532 2007-02-15 00:28 20_meta_tests.cf
-rw-r--r-- 1 root root 2350 2007-02-15 00:28 20_net_tests.cf
-rw-r--r-- 1 root root 16172 2007-02-15 00:28 20_phrases.cf
-rw-r--r-- 1 root root 5003 2007-02-15 00:28 20_porn.cf
-rw-r--r-- 1 root root 17065 2007-02-15 00:28 20_ratware.cf
-rw-r--r-- 1 root root 9901 2007-02-15 00:28 20_uri_tests.cf
-rw-r--r-- 1 root root 2520 2007-02-15 00:28 23_bayes.cf
-rw-r--r-- 1 root root 420 2007-02-15 00:28 25_accessdb.cf
-rw-r--r-- 1 root root 1534 2007-02-15 00:28 25_antivirus.cf
-rw-r--r-- 1 root root 9306 2007-02-15 00:28 25_body_tests_es.cf
-rw-r--r-- 1 root root 17865 2007-02-15 00:28 25_body_tests_pl.cf
-rw-r--r-- 1 root root 190 2007-02-15 00:28 25_dcc.cf
-rw-r--r-- 1 root root 2182 2007-02-15 00:28 25_dkim.cf
-rw-r--r-- 1 root root 2136 2007-02-15 00:28 25_domainkeys.cf
-rw-r--r-- 1 root root 2927 2007-02-15 00:28 25_hashcash.cf
-rw-r--r-- 1 root root 189 2007-02-15 00:28 25_pyzor.cf
-rw-r--r-- 1 root root 2201 2007-02-15 00:28 25_razor2.cf
-rw-r--r-- 1 root root 8531 2007-02-15 00:28 25_replace.cf
-rw-r--r-- 1 root root 3062 2007-02-15 00:28 25_spf.cf
-rw-r--r-- 1 root root 352 2007-02-15 00:28 25_textcat.cf
-rw-r--r-- 1 root root 6733 2007-02-15 00:28 25_uribl.cf
-rw-r--r-- 1 root root 47502 2007-02-15 00:28 30_text_de.cf
-rw-r--r-- 1 root root 34976 2007-02-15 00:28 30_text_fr.cf
-rw-r--r-- 1 root root 1856 2007-02-15 00:28 30_text_it.cf
-rw-r--r-- 1 root root 38124 2007-02-15 00:28 30_text_nl.cf
-rw-r--r-- 1 root root 30193 2007-02-15 00:28 30_text_pl.cf
-rw-r--r-- 1 root root 2878 2007-02-15 00:28 30_text_pt_br.cf
-rw-r--r-- 1 root root 33919 2007-02-15 00:28 50_scores.cf
-rw-r--r-- 1 root root 1302 2007-02-15 00:28 60_awl.cf
-rw-r--r-- 1 root root 5092 2007-02-15 00:28 60_whitelist.cf
-rw-r--r-- 1 root root 2532 2007-02-15 00:28 60_whitelist_dk.cf
-rw-r--r-- 1 root root 2556 2007-02-15 00:28 60_whitelist_dkim.cf
-rw-r--r-- 1 root root 3669 2007-02-15 00:28 60_whitelist_spf.cf
-rw-r--r-- 1 root root 1912 2007-02-15 00:28 60_whitelist_subject.cf
-rw-r--r-- 1 root root 939 2007-02-15 00:28 65_debian.cf
-rw-r--r-- 1 root root 101479 2007-02-15 00:28 languages
-rw-r--r-- 1 root root 3304 2007-02-15 00:28 sa-update-pubkey.txt
-rw-r--r-- 1 root root 18944 2007-02-15 00:28 triplets.txt
-rw-r--r-- 1 root root 1843 2007-02-15 00:28 user_prefs.template
####################################################################
Are these the files to be copied to ~/.spamassassin?
--
Thanks and regards,
Allen Williams
Office: +1.321.309.7931
Mobile: +1.321.258.1272
Re: Spamassassin Letting a Lot of Spams Through
Posted by aladdin <al...@csunv.com>.
On Saturday 13 September 2008 21:58, aladdin wrote:
> On Saturday 13 September 2008 20:38, aladdin wrote:
> > On Saturday 13 September 2008 20:30, Daryl C. W. O'Shea wrote:
> > > On 13/09/2008 8:20 PM, aladdin wrote:
> > > > On Saturday 13 September 2008 20:00, Daryl C. W. O'Shea wrote:
> > > >> Check to make sure that network tests aren't disabled. Many distro
> > > >> packages have network tests turned off my default. Not sure where
> > > >> Debian would configure this, sorry.
> > > >>
> > > >> Daryl
> > > >
> > > > Thanks for the reply!
> > > >
> > > > Where would I check that and what would I look for? Can you tell
> > > > that from either the header or the config file I posted?
> > >
> > > Not sure where Debian keeps its daemon config files, but you can
> > > probably find out by running the following command and looking for "-L"
> > > or "--local" in the output.
> > >
> > > ps aux | grep spamd
> > >
> > >
> > > Daryl
> >
> > Thanks again!
> >
> > Yeah, if you saw my last email, I checked that very thing. I believe
> > that all my config files are in /etc/spamassassin; that is where the
> > local.cf came from, and there are init.pre and v310, v312.... files in
> > there as well. That's where I looked to see if it appeared the networks
> > tests (razor, pyzor, etc.) where turned on, and they *appear* to be;-).
>
> A bit more data- lo & behold, I checked the log files, and here is what
> they say:
>
> #######################################################
> Sep 13 21:19:37 anw-dev spamd[17910]: bayes: cannot open bayes
> databases /home/anw/.spamassassin/bayes_* R/O: tie failed:
> Sep 13 21:20:37 anw-dev last message repeated 5 times
> Sep 13 21:24:41 anw-dev spamd[17910]: bayes: cannot open bayes
> databases /home/anw/.spamassassin/bayes_* R/O: tie failed:
> Sep 13 21:24:41 anw-dev last message repeated 2 times
> Sep 13 21:24:41 anw-dev spamd[17910]: bayes: cannot open bayes
> databases /home/anw/.spamassassin/bayes_* R/W: tie failed: Inappropriate
> ioctl for device
> Sep 13 21:35:55 anw-dev spamd[17910]: bayes: cannot open bayes
> databases /home/anw/.spamassassin/bayes_* R/O: tie failed:
> Sep 13 21:35:55 anw-dev last message repeated 2 times
> Sep 13 21:35:55 anw-dev spamd[17910]: bayes: cannot open bayes
> databases /home/anw/.spamassassin/bayes_* R/W: tie failed: Inappropriate
> ioctl for device
> Sep 13 21:39:57 anw-dev spamd[17910]: bayes: cannot open bayes
> databases /home/anw/.spamassassin/bayes_* R/O: tie failed:
> #########################################################
>
> So, evidently, it can't find my bayes database. So, since I want to use a
> system-wide database, where is it (/usr/share/spamassassin?, which has a
> lot of likely looking files in it), and how do I tell spamd to use it?
>
> This directory has a lot of bayes, razor, pyzor, etc. filenames in it, and
> this could be my problem.
Well, I have run (from the time of my last post) spamd with this command line:
/usr/sbin/spamd --create-prefs --max-children
5 --helper-home-dir=/usr/share/spamassassin -d --pidfile=/var/run/spamd.pid
and I still have the same problem with emails and the same log entries.
--
Thanks and regards,
Allen Williams
Office: +1.321.309.7931
Mobile: +1.321.258.1272
Re: Spamassassin Letting a Lot of Spams Through
Posted by aladdin <al...@csunv.com>.
On Saturday 13 September 2008 20:38, aladdin wrote:
> On Saturday 13 September 2008 20:30, Daryl C. W. O'Shea wrote:
> > On 13/09/2008 8:20 PM, aladdin wrote:
> > > On Saturday 13 September 2008 20:00, Daryl C. W. O'Shea wrote:
> > >> Check to make sure that network tests aren't disabled. Many distro
> > >> packages have network tests turned off my default. Not sure where
> > >> Debian would configure this, sorry.
> > >>
> > >> Daryl
> > >
> > > Thanks for the reply!
> > >
> > > Where would I check that and what would I look for? Can you tell that
> > > from either the header or the config file I posted?
> >
> > Not sure where Debian keeps its daemon config files, but you can
> > probably find out by running the following command and looking for "-L"
> > or "--local" in the output.
> >
> > ps aux | grep spamd
> >
> >
> > Daryl
>
> Thanks again!
>
> Yeah, if you saw my last email, I checked that very thing. I believe that
> all my config files are in /etc/spamassassin; that is where the local.cf
> came from, and there are init.pre and v310, v312.... files in there as
> well. That's where I looked to see if it appeared the networks tests
> (razor, pyzor, etc.) where turned on, and they *appear* to be;-).
A bit more data- lo & behold, I checked the log files, and here is what they
say:
#######################################################
Sep 13 21:19:37 anw-dev spamd[17910]: bayes: cannot open bayes
databases /home/anw/.spamassassin/bayes_* R/O: tie failed:
Sep 13 21:20:37 anw-dev last message repeated 5 times
Sep 13 21:24:41 anw-dev spamd[17910]: bayes: cannot open bayes
databases /home/anw/.spamassassin/bayes_* R/O: tie failed:
Sep 13 21:24:41 anw-dev last message repeated 2 times
Sep 13 21:24:41 anw-dev spamd[17910]: bayes: cannot open bayes
databases /home/anw/.spamassassin/bayes_* R/W: tie failed: Inappropriate
ioctl for device
Sep 13 21:35:55 anw-dev spamd[17910]: bayes: cannot open bayes
databases /home/anw/.spamassassin/bayes_* R/O: tie failed:
Sep 13 21:35:55 anw-dev last message repeated 2 times
Sep 13 21:35:55 anw-dev spamd[17910]: bayes: cannot open bayes
databases /home/anw/.spamassassin/bayes_* R/W: tie failed: Inappropriate
ioctl for device
Sep 13 21:39:57 anw-dev spamd[17910]: bayes: cannot open bayes
databases /home/anw/.spamassassin/bayes_* R/O: tie failed:
#########################################################
So, evidently, it can't find my bayes database. So, since I want to use a
system-wide database, where is it (/usr/share/spamassassin?, which has a lot
of likely looking files in it), and how do I tell spamd to use it?
This directory has a lot of bayes, razor, pyzor, etc. filenames in it, and
this could be my problem.
--
Thanks and regards,
Allen Williams
Office: +1.321.309.7931
Mobile: +1.321.258.1272
Re: Spamassassin Letting a Lot of Spams Through
Posted by aladdin <al...@csunv.com>.
On Saturday 13 September 2008 20:30, Daryl C. W. O'Shea wrote:
> On 13/09/2008 8:20 PM, aladdin wrote:
> > On Saturday 13 September 2008 20:00, Daryl C. W. O'Shea wrote:
> >> Check to make sure that network tests aren't disabled. Many distro
> >> packages have network tests turned off my default. Not sure where
> >> Debian would configure this, sorry.
> >>
> >> Daryl
> >
> > Thanks for the reply!
> >
> > Where would I check that and what would I look for? Can you tell that
> > from either the header or the config file I posted?
>
> Not sure where Debian keeps its daemon config files, but you can
> probably find out by running the following command and looking for "-L"
> or "--local" in the output.
>
> ps aux | grep spamd
>
>
> Daryl
Thanks again!
Yeah, if you saw my last email, I checked that very thing. I believe that all
my config files are in /etc/spamassassin; that is where the local.cf came
from, and there are init.pre and v310, v312.... files in there as well.
That's where I looked to see if it appeared the networks tests (razor, pyzor,
etc.) where turned on, and they *appear* to be;-).
--
Thanks and regards,
Allen Williams
Office: +1.321.309.7931
Mobile: +1.321.258.1272
Re: Spamassassin Letting a Lot of Spams Through
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 13/09/2008 8:20 PM, aladdin wrote:
> On Saturday 13 September 2008 20:00, Daryl C. W. O'Shea wrote:
>> Check to make sure that network tests aren't disabled. Many distro
>> packages have network tests turned off my default. Not sure where
>> Debian would configure this, sorry.
>>
>> Daryl
>
> Thanks for the reply!
>
> Where would I check that and what would I look for? Can you tell that from
> either the header or the config file I posted?
Not sure where Debian keeps its daemon config files, but you can
probably find out by running the following command and looking for "-L"
or "--local" in the output.
ps aux | grep spamd
Daryl
Re: Spamassassin Letting a Lot of Spams Through
Posted by aladdin <al...@csunv.com>.
On Saturday 13 September 2008 20:00, Daryl C. W. O'Shea wrote:
> Check to make sure that network tests aren't disabled. Many distro
> packages have network tests turned off my default. Not sure where
> Debian would configure this, sorry.
>
> Daryl
Thanks for the reply!
Where would I check that and what would I look for? Can you tell that from
either the header or the config file I posted?
--
Thanks and regards,
Allen Williams
Office: +1.321.309.7931
Mobile: +1.321.258.1272
Re: Spamassassin Letting a Lot of Spams Through
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
Check to make sure that network tests aren't disabled. Many distro
packages have network tests turned off my default. Not sure where
Debian would configure this, sorry.
Daryl
Re: Spamassassin Letting a Lot of Spams Through
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 13.09.08 19:44, aladdin Sorry about the generic wrote:
> Sorry about the generic subject, but it is the only thing this newbie knows to
> describe the symptom.
>
> Platform: Debian (Etch?)
>
> Latest Spamassassin in apt (version 3.1.7-deb)
there's 3.2.3 in volatile archive, just FYI
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Emacs is a complicated operating system without good text editor.
Re: Spamassassin Letting a Lot of Spams Through
Posted by Danita Zanre <da...@caledonia.net>.
>>> Matus UHLAR - fantomas <uh...@fantomas.sk> 9/22/2008 3:26 AM >>>
On 13.09.08 19:44, aladdin Sorry about the generic wrote:
> Sorry about the generic subject, but it is the only thing this newbie
knows to
> describe the symptom.
I have used spamassassin for many years. We use a number of add-ins
like Razor2, botnet checks, selective greylisting before hitting
spamassassin, etc., and we too have been seeing an increase in "leakage"
lately. Just this weekend I increased our Bayes scores. We had a slew
of mail in the 4.5-5.00 range (we block at 5.00) that was spam, but also
some real mail, so I did not want to lower the trigger score. Changing
the 95% bayes to 4.5 from 3.5 has helped tremendously. I don't see any
false positives in this, but we're blocking the majority of that 4.5-5.0
scored spam of course, because the vast majority of it was high bayes,
but very few other hits.
Danita