You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Chris Brandstetter <cb...@nebrwesleyan.edu> on 2014/07/31 21:59:51 UTC
Problem with rules not being parsed I think
We are running spam assassin 3.3.1 with Postfix. So I have added some rules to mark as spam yolasite, wix, etc…. as we have been getting besieged by them. My thing is these rules seem not to be loading. When I do a spamassassin —lint I don’t get any errors. When I do spamassassin —lint -D I get pages of output that I have a hard time sifting through, but nothing more than a warning. Here are the config file contents etc…. Any help would be HUGELY appreciated.
New Rules for our local.cf:
#
# yolasite.com BAH BYE!!!!
body NWU_YOLASITE /yolasite\.com/i
describe NWU_YOLASITE yolasite.com hosts LOTS of phishing attacks
score NWU_YOLASITE 12
#
# jimdo.com BAH BYE!!!!
body NWU_JIMDO /jimdo\.com/i
describe NWU_JIMDO jimdo.com hosts LOTS of phishing attacks
score NWU_JIMDO 12
#
# wix.com BAH BYE!!!!
body NWU_WIX /wix\.com/i
describe NWU_WIX wix.com hosts LOTS of phishing attacks
score NWU_WIX 12
#
# .ru/ links in the body, pump to a 4
body NWU_RUDOMAIN /\.ru\//i
describe NWU_RUDOMAIN ru links in the body are most likely phasing attacks
score NWU_RUDOMAIN 3.5
#
# Yolasite URI rule
uri NWU_URI_YOLASITE /\.yolasite\.com\//
describe NWU_URI_YOLASITE yolasite.com hosts LOTS of phishing attacks
score NWU_URI_YOLASITE 12
/etc/postfix/master.cf
flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp unix - n n - - pipe
flags=Fq. user=foo argv=/usr/local/sbin/bsmtp -f $sender $nexthop $recipient
#
# see recipient_access for spam scanning rules
spamassassin unix - n n - - pipe
user=spamd argv=/usr/bin/spamc -u spamd -e
/usr/sbin/sendmail -oi -f ${sender} ${recipient}
# AV scan filter (used by content_filter)
scan unix - - n - 16 smtp
-o smtp_send_xforward_command=yes
# For injecting mail back into postfix from clamSMTP
127.0.0.1:10026 inet n - n - 16 smtpd
-o content_filter=
-o smtpd_proxy_filter=
-o receive_override_options=no_unknown_recipient_checks,no_header_body_checks,no_milters
-o smtpd_helo_restrictions=
-o smtpd_client_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_authorized_xforward_hosts=127.0.0.0/8
-o mynetworks=127.0.0.0/8
-o smtpd_recipient_restrictions=permit_mynetworks,reject
Existing /etc/mail/spamassassin/local.cf
# use bayesian filtering
use_bayes 1
# train bayes
bayes_auto_learn 1
bayes_auto_expire 1
# Bayesean database location. Store it in SQL so we can share it between
# the two machines, and do per-user bayes
bayes_store_module Mail::SpamAssassin::BayesStore::MySQL
bayes_sql_dsn DBI:mysql:spamassassin:db
bayes_sql_username $username
bayes_sql_password $password
# autowhitelist database location.
auto_whitelist_factory Mail::SpamAssassin::SQLBasedAddrList
user_awl_dsn DBI:mysql:spamassassin:db
user_awl_sql_username $username
user_awl_sql_password $password
user_awl_sql_table awl
# Use Razor2 (newly free!) and Pyzor (always free!). Don't use DCC because it sucks
#use_razor2 1
#razor_config /etc/razor/razor-agent.conf
use_pyzor 1
pyzor_options --homedir /var/lib/pyzord
# don't block apparent backscatter from ourselves
# If you are sure you have DNS access set it to "yes".
dns_available yes
required_hits 5
report_safe 0
clear_headers
add_header spam Flag _YESNOCAPS_
add_header all Level _STARS(*)_
add_header all Status _YESNO_, bayes=_BAYES_ score=_SCORE_ required=_REQD_ tests=_TESTSSCORES(,)_ autolearn=_AUTOLEARN_ version=_VERSION_
rewrite_header Subject [SPAM:_STARS(*)_]
internal_networks 10.0.0.0/8
# Yahoo! lists tend to get blocked because they put spam, er, ads in them
whitelist_from_rcvd *.mail.mud.yahoo.com *.bullet.scd.yahoo.com
# BigPulse for SAS online voting
whitelist_from_rcvd *@bigpulse.com *.bigpulse.com
####################### SCORE ADJUSTMENTS ######################
# For default scores have a look at /usr/local/share/spamassassin/50_scores.cf
# file.
score HTML_IMAGE_RATIO_02 2.5
score HTML_IMAGE_ONLY_04 3.5
score HTML_IMAGE_ONLY_08 3
score DEAR_FRIEND 2
score DEAR_WINNER 2.5
score FORGED_HOTMAIL_RCVD2 2
score UNPARSEABLE_RELAY 0.5
score MPART_ALT_DIFF 0.5
score BAD_CREDIT 2.5
score NORMAL_HTTP_TO_IP 0.75
score SUBJ_ALL_CAPS 2
score EXTRA_MPART_TYPE 1.5
score SARE_EN_A_1XX_1 2.5
score SARE_EN_A_1XX_2 2.5
score MICROSOFT_EXECUTABLE 2
score US_DOLLARS_3 1.5
# this rule is broken
score FH_DATE_PAST_20XX 0.1
# these tests are good, but be careful about turning up the scores
# for the lower ones
score ADVANCE_FEE_2 1.75
score ADVANCE_FEE_3 3.75
score ADVANCE_FEE_4 4.25
# these catch a lot of animated-GIF stock spam
score SARE_GIF_ATTACH 2
score SARE_GIF_STOX 2.5
# crank up checksum systems -- that's been manually verified as spam
score PYZOR_CHECK 5
score RAZOR2_CHECK 6
score DIGEST_MULTIPLE 2 # hit > 1 checksum system
# bayesian scores
score BAYES_00 -1.5
score BAYES_05 -0.5
score BAYES_20 0.25
score BAYES_40 0.75
score BAYES_50 1
score BAYES_60 1.5
score BAYES_80 2
score BAYES_95 3
score BAYES_99 3.5
# these are hitting *a lot* on spam. If your message contains a blacklisted
# URL, you're toast.
score URIBL_BLACK 12
score URIBL_SBL 12
score URIBL_JP_SURBL 12
score URIBL_OB_SURBL 12
score URIBL_WS_SURBL 12
score URIBL_SC_SURBL 12
score URIBL_AB_SURBL 12
# false positives on messages in Chinese, Japanese, and other languages
# that don't delineate words with spaces
score TVD_SPACE_RATIO 0.1
# lots of FPs recently -- no idea why.
score EMPTY_MESSAGE 0.1
################################################
# CUSTOM RULES #
################################################
# DEAR_FRIEND is good, but some people send to "Dear good friend", etc.
body NWU_DEAR_X_FRIEND /^\s*Dear \w+ Friend\b/i
describe NWU_DEAR_X_FRIEND Dear *** Friend -- catches adjectives in between
score NWU_DEAR_X_FRIEND 1.5
# replace the normal "big money" rule with several that are more sensitive
score BILLION_DOLLARS 0
# big money with more than just a space between
body NWU_BILLION_DOLLARS_OBFU_SPACE /[bm]illion.{0,5}dollar/i
describe NWU_BILLION_DOLLARS_OBFU_SPACE Talks about lots of money
score NWU_BILLION_DOLLARS_OBFU_SPACE 1
# big money match with country name
body NWU_BILLION_US_DOLLARS /[bm]illion.?(U\.S\.|US|United States).?dollar/i
describe NWU_BILLION_US_DOLLARS Talks about lots of American money
score NWU_BILLION_US_DOLLARS 1
# british big money match
body NWU_BILLION_POUNDS /[bm]illion.pound/i
describe NWU_BILLION_POUNDS Talks about lots of British money
score NWU_BILLION_POUNDS 0.5
# british big money match
body NWU_BILLION_GB_POUNDS /([bm]illion|thousand).?(british|great.britain).?pound/i
describe NWU_BILLION_GB_POUNDS Talks about lots of British money
score NWU_BILLION_GB_POUNDS 1
# canadian medication
body NWU_CANADIAN_MEDS /canadian (med|medication)s?/i
describe NWU_CANADIAN_MEDS Canadian medication, meds, etc.
score NWU_CANADIAN_MEDS 1.5
# mentions the CAN-SPAM act, a pretty good sign that they ARE-SPAM
body NWU_CAN_SPAM /can.spam act/i
describe NWU_CAN_SPAM Mentions the CAN-SPAM Act
score NWU_CAN_SPAM 2
# sending server has no PTR record
header NWU_RCVD_INVALID_PTR2 Received =~ /from \S+ \(unknown /
describe NWU_RCVD_INVALID_PTR2 Server in "Received" header has no PTR
score NWU_RCVD_INVALID_PTR2 1.5
# sender isn't sure if you're male or female -- used a lot in 409s
body NWU_SIR_MADAM /sir.{0,4}madam/i
describe NWU_SIR_MADAM Sender addresses letter to "Sir/Madam", "Sir or Madam", or other with obfuscated separator
score NWU_SIR_MADAM 1
# from yahoo.com.blah (i.e., foreign Yahoo! account)
header NWU_FOREIGN_YAHOO From =~ /\@yahoo(?:\.com?)?\.\w\w$/i
describe NWU_FOREIGN_YAHOO Sender address is from foreign Yahoo! account
score NWU_FOREIGN_YAHOO 1.5
# claims you have no financial obligation
body NWU_NO_OBLIGATION /no (financial)? obligation/i
describe NWU_NO_OBLIGATION Claims you have no (financial) obligation
score NWU_NO_OBLIGATION 1.5
# if a message hits FORGED_OUTLOOK_HTML, it's pretty much guaranteed to hit
# FORGED_MUA_OUTLOOK. That means that one rule (basically) is worth 7 points,
# which is way too much. Don't punish these messages that badly.
meta NWU_FORGED_OUTLOOK FORGED_OUTLOOK_HTML && FORGED_MUA_OUTLOOK
describe NWU_FORGED_OUTLOOK Message hit 2 redundant Outlook rules
score NWU_FORGED_OUTLOOK -3
# a lot of image spam is sent as multipart/related instead of multipart/mixed
header NWU_MULTIPART_RELATED Content-Type =~ /multipart\/related;/
describe NWU_MULTIPART_RELATED Stock messages with image attachments
score NWU_MULTIPART_RELATED 0.5
# mentions Lincoln, Omaha, or Nebraska
body NWU_LOCAL_PLACES /Lincoln|Omaha|Nebraska/i
describe NWU_LOCAL_PLACES Mentions Lincoln, Omaha, or Nebraska
score NWU_LOCAL_PLACES -5
# lets you know that you're a winner
body NWU_WINNER_NOTIFY /winner notification/i
describe NWU_WINNER_NOTIFY Lets you know that you are a winner
score NWU_WINNER_NOTIFY 0.75
# French lottery (thank you, French colonialism in Africa)
body NWU_FRENCH_LOTTERY /tombola|loterie/i
describe NWU_FRENCH_LOTTERY Message about a lottery in French
score NWU_FRENCH_LOTTERY 0.5
# "add .com after the dot"
body NWU_OBFU_DOMAIN1 /\bwww[^a-z].{0,30}(?:Replace.{1,4}\bwith\b|\badd\b(?:com|net|org)\b|\bafter dot\b)/i
describe NWU_OBFU_DOMAIN1 Obfuscated domain and request to fix it
score NWU_OBFU_DOMAIN1 1.5
# block nasty phishing attempt
body NWU_REFUSES_TO_UPDATE /Account (user|owner)s? that refuses? to update (their|his.{0,4}her) account/
describe NWU_REFUSES_TO_UDPATE Common text from many phishing attempts
score NWU_REFUSES_TO_UPDATE 5
# broken outlook forgery
header _NWU_MSGID_OUTLOOK_888 Message-Id =~ /^<[0-9a-f]{8}(?:\$[0-9a-f]{8}){2}\@/
score _NWU_MSGID_OUTLOOK_888 0.1
header _NWU_OUTLOOK_MUA X-Mailer =~ /^Microsoft (?:Office )?Outlook\b/
score _NWU_OUTLOOK_MUA 0.1
meta NWU_RATWARE_MSGID _NWU_MSGID_OUTLOOK_888 && _NWU_OUTLOOK_MUA
describe NWU_RATWARE_MSGID Ratware Message-Id
score NWU_RATWARE_MSGID 3
header NWU_FREE_REPLY_TO Reply-To =~ /\@(gmail\.com|hotmail\.co[\.m]|yahoo\.co[\.m]|live\.com|mcom.com)/i
describe NWU_FREE_REPLY_TO Message has a Reply-To at a popular free mail host
score NWU_FREE_REPLY_TO 3
header NWU_DOMAIN_FORGERY X-Domain-Forgery =~ /Yes/
describe NWU_DOMAIN_FORGERY Sender forged our domain in the envelope sender
score NWU_DOMAIN_FORGERY 3
# phishers often use "surport" instead of "support" in their email addresses
header NWU_FROM_SURPORT From =~ /surport/i
describe NWU_FROM_SURPORT Messages has From: including "surport"
score NWU_FROM_SURPORT 1
header NWU_REPLY_TO_SURPORT Reply-To =~ /surport/i
describe NWU_REPLY_TO_SURPORT Messages has Reply-To: including "surport"
score NWU_REPLY_TO_SURPORT 2
header NWU_DOMAIN_FORGERY X-NWU-Sender-Karma =~ /hi/
describe NWU_DOMAIN_FORGERY Sender is on many whitelists/address books
score NWU_DOMAIN_FORGERY -3
header NWU_DOMAIN_FORGERY X-NWU-Sender-Karma =~ /med/
describe NWU_DOMAIN_FORGERY Sender is on some whitelists/address books
score NWU_DOMAIN_FORGERY -1
header NWU_DOMAIN_FORGERY X-NWU-Sender-Karma =~ /low/
describe NWU_DOMAIN_FORGERY Sender is on a mix of whitelists/blacklists
score NWU_DOMAIN_FORGERY 0.1
header NWU_DOMAIN_FORGERY X-NWU-Sender-Karma =~ /none/
describe NWU_DOMAIN_FORGERY Sender is on many blacklists
score NWU_DOMAIN_FORGERY 1
header NWU_DOMAIN_FORGERY X-NWU-Domain-Karma =~ /hi/
describe NWU_DOMAIN_FORGERY Domain is on many whitelists/address books
score NWU_DOMAIN_FORGERY -1
header NWU_DOMAIN_FORGERY X-NWU-Domain-Karma =~ /med/
describe NWU_DOMAIN_FORGERY Domain is on some whitelists/address books
score NWU_DOMAIN_FORGERY -0.5
header NWU_DOMAIN_FORGERY X-NWU-Domain-Karma =~ /low/
describe NWU_DOMAIN_FORGERY Domain is on a mix of whitelists/blacklists
score NWU_DOMAIN_FORGERY 0.1
header NWU_DOMAIN_FORGERY X-NWU-Domain-Karma =~ /none/
describe NWU_DOMAIN_FORGERY Domain is on many blacklists
score NWU_DOMAIN_FORGERY 0.5
Chris Brandstetter
System Administrator
Nebraska Wesleyan University
⌘
Re: Problem with rules not being parsed I think
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Thu, 2014-07-31 at 19:59 +0000, Chris Brandstetter wrote:
> body NWU_YOLASITE /yolasite\.com/i
This (and the other similar ones) really should better be a uri rule.
And this particular one be merged with the almost duplicating uri rule
just 20 lines below.
> # Yolasite URI rule
> uri NWU_URI_YOLASITE /\.yolasite\.com\//
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Problem with rules not being parsed I think
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 7/31/2014 7:29 PM, Karsten Bräckelmann wrote:
> Moreover, it is important to note that this debug warning was
> introduced in 3.4. In particular, since the OP is running 3.3.x. ;)
And you didn't beat him up for running an outdated version? For shame!
Regards,
KAM
Re: Problem with rules not being parsed I think
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Thu, 2014-07-31 at 19:02 -0400, Kevin A. McGrail wrote:
> On 7/31/2014 5:41 PM, Karsten Bräckelmann wrote:
> > A missing description is not an error, not even a lint warning.
>
> Yes, it is a warning (though in my defense I was using "error"
> generically.
Hah, that's actually news to me!
However, that is merely a debug message, phrased to look like a warning.
It is not an actual lint warning or generic error. Without the -D
switch, the command below returns nothing. A plain lint check also exits
clean. (In contrast, a describe option for an undefined rule does
trigger a lint warning.)
Moreover, it is important to note that this debug warning was introduced
in 3.4. In particular, since the OP is running 3.3.x. ;)
> I think this explains why some of your rules are missing
> descriptions ;-)
Doh!
> spamassassin -D --lint 2>&1 | grep -i description | grep KB
> Jul 31 19:01:36.329 [31451] dbg: config: warning: no description set for
> KB_RATWARE_MSGID
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Problem with rules not being parsed I think
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 7/31/2014 5:41 PM, Karsten Bräckelmann wrote:
> On Thu, 2014-07-31 at 17:34 -0400, Kevin A. McGrail wrote:
>> Put it in a different CF and comment out a description and look for an
>> error that a description is missing.
> A missing description is not an error, not even a lint warning.
>
>
Yes, it is a warning (though in my defense I was using "error"
generically. I think this explains why some of your rules are missing
descriptions ;-)
spamassassin -D --lint 2>&1 | grep -i description | grep KB
Jul 31 19:01:36.329 [31451] dbg: config: warning: no description set for
KB_RATWARE_MSGID
Jul 31 19:01:36.362 [31451] dbg: config: warning: no description set for
KB_RATWARE_OUTLOOK_08
Jul 31 19:01:36.369 [31451] dbg: config: warning: no description set for
KB_RATWARE_OUTLOOK_MID
Jul 31 19:01:36.379 [31451] dbg: config: warning: no description set for
KB_FAKED_THE_BAT
Jul 31 19:01:36.410 [31451] dbg: config: warning: no description set for
KB_RATWARE_OUTLOOK_16
Jul 31 19:01:36.418 [31451] dbg: config: warning: no description set for
KB_DATE_CONTAINS_TAB
Jul 31 19:01:36.421 [31451] dbg: config: warning: no description set for
KB_RATWARE_OUTLOOK_12
Jul 31 19:01:36.424 [31451] dbg: config: warning: no description set for
KB_RATWARE_BOUNDARY
regards,
KAM
Re: Problem with rules not being parsed I think
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Thu, 2014-07-31 at 17:34 -0400, Kevin A. McGrail wrote:
> Put it in a different CF and comment out a description and look for an
> error that a description is missing.
A missing description is not an error, not even a lint warning.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Problem with rules not being parsed I think
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 7/31/2014 5:08 PM, Chris Brandstetter wrote:
> If showed it being loaded "Jul 31 15:43:07.794 [20903] dbg: config:
> read file /etc/mail/spamassassin/local.cf” Is there any way to list
> what rules are being loaded? Like a list all available rules type of
> thing?
Not that I'm aware of, no. Put it in a different CF and comment out a
description and look for an error that a description is missing.
Re: Problem with rules not being parsed I think
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Thu, 2014-07-31 at 21:08 +0000, Chris Brandstetter wrote:
> If showed it being loaded "Jul 31 15:43:07.794 [20903] dbg: config:
> read file /etc/mail/spamassassin/local.cf” Is there any way to list
> what rules are being loaded? Like a list all available rules type of
> thing?
No, but you can easily test for it. All you need is the string your rule
should match, wrapped in a minimal "message". That is, completely empty
mail headers, an empty line as delimiter, and the actual string as the
plain text body.
echo -e "\n test string" | spamassassin
Fed to spamassassin, the rule should be included in the resulting X-Spam
headers.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Problem with rules not being parsed I think
Posted by Chris Brandstetter <cb...@nebrwesleyan.edu>.
If showed it being loaded "Jul 31 15:43:07.794 [20903] dbg: config: read file /etc/mail/spamassassin/local.cf” Is there any way to list what rules are being loaded? Like a list all available rules type of thing?
Chris Brandstetter
System Administrator
Nebraska Wesleyan University
Office Phone: +1 (402)465-7549
On Campus: 7549
⌘
On Jul 31, 2014, at 3:18 PM, Kevin A. McGrail <KM...@PCCC.com> wrote:
> On 7/31/2014 3:59 PM, Chris Brandstetter wrote:
>> We are running spam assassin 3.3.1 with Postfix. So I have added some rules to mark as spam yolasite, wix, etc…. as we have been getting besieged by them. My thing is these rules seem not to be loading. When I do a spamassassin —lint I don’t get any errors. When I do spamassassin —lint -D I get pages of output that I have a hard time sifting through, but nothing more than a warning. Here are the config file contents etc…. Any help would be HUGELY appreciated.
> Put the rules in a file such as chris.cf in the same dir.
>
> Run spamassassin -D --lint 2>&1 | grep -i chris to see if the file is loaded.
>
> Then, are you restarting spamd to reload the cf file?
>
> Regards,
> KAM
Re: Problem with rules not being parsed I think
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 7/31/2014 3:59 PM, Chris Brandstetter wrote:
> We are running spam assassin 3.3.1 with Postfix. So I have added some
> rules to mark as spam yolasite, wix, etc…. as we have been getting
> besieged by them. My thing is these rules seem not to be loading.
> When I do a spamassassin —lint I don’t get any errors. When I do
> spamassassin —lint -D I get pages of output that I have a hard time
> sifting through, but nothing more than a warning. Here are the config
> file contents etc…. Any help would be HUGELY appreciated.
Put the rules in a file such as chris.cf in the same dir.
Run spamassassin -D --lint 2>&1 | grep -i chris to see if the file is
loaded.
Then, are you restarting spamd to reload the cf file?
Regards,
KAM