You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Shane Williams <sh...@shanew.net> on 2016/09/08 20:53:00 UTC
RCVD_IN_SORBS_SPAM and google IPs
Hey all,
I'm seeing google IP ranges hit the RCVD_IN_SORBS_SPAM rule, and in
digging deeper, I realize that there are zero hits on this rule for
the two weeks prior to Aug. 31, and now I'm seeing it thousands of
times per week (not just against google IPs).
Was this rule added/changed/re-scored in a recent sa-update? I looked
at ruleqa.spamassassin.org, and just at a glance notice that the rule
doesn't seem to be in commits previous to Aug. 30, but I may totally
be reading the site's information wrong.
I've turned the score down to a tiny, but non-zero value for now,
because it seems to be pushing legit emails close (if not over) the
local threshold.
--
Public key #7BBC68D9 at | Shane Williams
http://pgp.mit.edu/ | System Admin - UT CompSci
=----------------------------------+-------------------------------
All syllogisms contain three lines | shanew@shanew.net
Therefore this is not a syllogism | www.ischool.utexas.edu/~shanew
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by RW <rw...@googlemail.com>.
On Sat, 17 Jun 2017 08:55:48 -0700 (MST)
hmiller wrote:
> Hi,
>
> Commonly RCVD_IN_ rules are checking the last untrusted relay,
Most positive scoring rules check the last-external.
> but
> RCVD_IN_SORBS_WEB is apparently doing all Received hops.
>
> Received: from host (host [2.2.2.2]) #The last untrusted relay
> Received: from [192.168.1.100] ([1.1.1.1]) #Authenticated MUA
>
> I would expect it to check only 2.2.2.2 (the last untrusted hop), but
> in this case 1.1.1.1 was listed in SORBS_WEB and was scored 1.50.
In theory it seems reasonable:
describe RCVD_IN_SORBS_WEB SORBS: sender is an abusable web server
An abused web-server may relay through a separate mail server. And
since web servers usually have static addresses it shouldn't be a
problem to do a deep check.
However in this case it looks like a dynamic address has got into list.
If this is common, it may be necessary to make it last-external.
It may be worth creating a separate last-external rule and see what
happens to the scores.
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by hmiller <he...@gmx.com>.
Hi,
Commonly RCVD_IN_ rules are checking the last untrusted relay, but
RCVD_IN_SORBS_WEB is apparently doing all Received hops.
Received: from host (host [2.2.2.2]) #The last untrusted relay
Received: from [192.168.1.100] ([1.1.1.1]) #Authenticated MUA
I would expect it to check only 2.2.2.2 (the last untrusted hop), but in
this case 1.1.1.1 was listed in SORBS_WEB and was scored 1.50.
--
View this message in context: http://spamassassin.1065346.n5.nabble.com/RCVD-IN-SORBS-SPAM-and-google-IPs-tp122542p134044.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by "Zinski, Steve" <sz...@richmond.edu>.
I’m seeing the same thing here, I’ve had to adjust that score lower. Also seeing lots of RCVD_IN_SORBS_WEB false-positives.
On 9/8/16, 4:53 PM, "Shane Williams" <sh...@shanew.net> wrote:
Hey all,
I'm seeing google IP ranges hit the RCVD_IN_SORBS_SPAM rule, and in
digging deeper, I realize that there are zero hits on this rule for
the two weeks prior to Aug. 31, and now I'm seeing it thousands of
times per week (not just against google IPs).
Was this rule added/changed/re-scored in a recent sa-update? I looked
at ruleqa.spamassassin.org, and just at a glance notice that the rule
doesn't seem to be in commits previous to Aug. 30, but I may totally
be reading the site's information wrong.
I've turned the score down to a tiny, but non-zero value for now,
because it seems to be pushing legit emails close (if not over) the
local threshold.
--
Public key #7BBC68D9 at | Shane Williams
http://pgp.mit.edu/ | System Admin - UT CompSci
=----------------------------------+-------------------------------
All syllogisms contain three lines | shanew@shanew.net
Therefore this is not a syllogism | www.ischool.utexas.edu/~shanew
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by David Jones <dj...@ena.com>.
>From: lists@rhsoft.net <li...@rhsoft.net>
>Sent: Monday, September 12, 2016 12:13 PM
>To: users@spamassassin.apache.org
>Subject: Re: RCVD_IN_SORBS_SPAM and google IPs
>that's exactly what i *don't* have a contentfilter for to need customers
>report their spam and i have to talk with abuse departments to stop it
I didn't have to do anything but check my logs to see that it stopped.
>> From my experience, they are trustworthy and police their
>> outbound spam properly to trust. Otherwise you will block too much
>> legit email from their Simple Email Service.
>why should i block too much legit mail just because a sender is not
>whitelisted?
It is possible for your Bayes to score high on a legit email and block it.
I don't have hours each week to manually train my Bayes database so
I had to find something more "hands off" to augment my pretty good,
not perfect, Bayes database.
DCC and RAZOR can sometimes push legit email over the block threshold
too.
MailSpike can be a bit too aggressive sometimes.
My main point was that there are different tiers of senders so if you
see patterns in your logs you can safely whitelist many senders in the
top tier using shortcircuit. Each person has to go by their logs and
find those in the top tier senders and then add them to their
whitelist_auth list which will help with issues like the subject of this
thread.
>> https://aws.amazon.com/ses/faqs/
>> They have sending and bounce quotas which are going to catch most
>> bad actors using SES.
>>
>>>the same for "whitelist_auth *@icloud.com"
>>
>> Apple is also doing a good job of policing their outbound spam
>> from icloud.com. My logs show good reputation of the IPs.
>> senderscore.org report for 17.164.24.103 has a 98 out of 100
>> as a very high sender which is excellent.
>a good job don't help much in case of hacked accounts which are closed
>after the damage of sending phising and malware mails already happened
There's not much anything can do to block compromised accounts from
normally trusted mail servers. Greylisting can help a little but not a lot.
>> Those are
>> message IDs which wouldn't necessarily match the envelope-from
>> used by whitelist_auth.
>man i know what i am doing when reading maillogs (besides i knew before
>looking in the recent logs that @amazonses.com is not a blindly
>trustable envelope)
Ok. I understand you know what you are doing now but I didn't know for
sure based on your log paste.
>from=<01...@amazonses.com>
>from a8-21.smtp-out.amazonses.com[54.240.8.21]
>> I don't see an IP address either to check the
>> source so that email could have been forwarded
>* SPF_PASS
>* DKIM_VALID_AU
You didn't show the envelope-from the first time so I couldn't
have assumed the hits above weren't forwarded. SPF_PASS and
DKIM_VALID_AU hits don't prove I was incorrect without showing
the original envelope-from.
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by "lists@rhsoft.net" <li...@rhsoft.net>.
Am 12.09.2016 um 18:53 schrieb David Jones:
> *>From:*lists@rhsoft.net <li...@rhsoft.net>
>
> *>Sent:* Monday, September 12, 2016 8:47 AM
> *>To:* users@spamassassin.apache.org
> *>Subject:* Re: RCVD_IN_SORBS_SPAM and google IPs
>
>>Am 12.09.2016 um 15:37 schrieb David Jones:
>>>>Has RCVD_IN_SORBS_WEB been considered for adjustment as well? It's
>>>>hitting a lot more ham than spam here, including mail from facebook.
>>>
>>> You should be safely whitelisting any major senders like Facebook at
>>> the MTA level and in SA:
>>>
>>> whitelist_auth *@amazonses.com
>>for sure *not* since that would whitelist anything hosted on the amazon
>>cloud instances which is *not* amazon stuff itself
>
>>don't confuse major good senders with hosted crap of endcustomers
>>@amazonses.com != @amazon.com
>
> I know the difference between amazonses.com and amazon.com. I have
> only had 1 instance of spam from amazonses.com and Amazon blocked
> it quickly.
that's exactly what i *don't* have a contentfilter for to need customers
report their spam and i have to talk with abuse departments to stop it
> From my experience, they are trustworthy and police their
> outbound spam properly to trust. Otherwise you will block too much
> legit email from their Simple Email Service.
why should i block too much legit mail just because a sender is not
whitelisted?
> https://aws.amazon.com/ses/faqs/
> They have sending and bounce quotas which are going to catch most
> bad actors using SES.
>
>>the same for "whitelist_auth *@icloud.com"
>
> Apple is also doing a good job of policing their outbound spam
> from icloud.com. My logs show good reputation of the IPs.
> senderscore.org report for 17.164.24.103 has a 98 out of 100
> as a very high sender which is excellent.
a good job don't help much in case of hacked accounts which are closed
after the damage of sending phising and malware mails already happened
> Everyone doesn't have to whitelist_auth the same senders. I only
> wanted to show that this is a valid way to reduce false positives
> for transient things like Google IPs in SORBS RBL.
>
>>[root@mail-gw:~]$ cat maillog | grep
>>01000157007004fc-dd484ffc-155c-48dc-8a7d-b9fbc51b7094-000000@email.amazonses.com
>>Sep 6 18:58:47 mail-gw postfix/cleanup[5554]: 3sTCVH11mDz9bQ:
>>message-id=<01...@email.amazonses.com>
>>Sep 6 18:58:52 mail-gw spamd[1086]: spamd: result: Y 14 -
>>BAYES_99,BAYES_999,BOGOFILTER_SPAM,CUST_DNSBL_19_SPAMCANN,CUST_DNSWL_5_ORG_N,CUST_DNSWL_8_TL_N,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_D>OMAINS,HTML_MESSAGE,RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,RCVD_IN_MSPIKE_H2,RP_MATCHES_RCVD,SPF_PASS,T_OBFU_ATTACH_MISSP,URIBL_LOC>AL
>>scantime=5.0,size=13908,user=sa-milt,uid=189,required_score=5.5,rhost=localhost,raddr=127.0.0.1,rport=/run/spamassassin/spamassassin.sock,mid=<01...@email.amazonses.com>,bayes=1.000000,autolearn=disabled,shortcircuit=no
>
> Did you check the envelope-from address of that message?
surely - in fact i found the message-id after grep for envelopes in
milter-reject-log and *then* seeked for the classification
> Those are
> message IDs which wouldn't necessarily match the envelope-from
> used by whitelist_auth.
man i know what i am doing when reading maillogs (besides i knew before
looking in the recent logs that @amazonses.com is not a blindly
trustable envelope)
from=<01...@amazonses.com>
from a8-21.smtp-out.amazonses.com[54.240.8.21]
> I don't see an IP address either to check the
> source so that email could have been forwarded
* SPF_PASS
* DKIM_VALID_AU
> I would need to see
> the full headers and the message body since it did hit so many rules
> and high Bayes
no you don't - the destination of that mail was our sysadmin-address
*never* used for subscribe anywhere nor as envelope-sender, it's only
mentioned on http error pages (non 2xx response)
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by David Jones <dj...@ena.com>.
>From: lists@rhsoft.net <li...@rhsoft.net>
>Sent: Monday, September 12, 2016 8:47 AM
>To: users@spamassassin.apache.org
>Subject: Re: RCVD_IN_SORBS_SPAM and google IPs
>Am 12.09.2016 um 15:37 schrieb David Jones:
>>>Has RCVD_IN_SORBS_WEB been considered for adjustment as well? It's
>>>hitting a lot more ham than spam here, including mail from facebook.
>>
>> You should be safely whitelisting any major senders like Facebook at
>> the MTA level and in SA:
>>
>> whitelist_auth *@amazonses.com
>for sure *not* since that would whitelist anything hosted on the amazon
>cloud instances which is *not* amazon stuff itself
>don't confuse major good senders with hosted crap of endcustomers
>@amazonses.com != @amazon.com
I know the difference between amazonses.com and amazon.com. I have
only had 1 instance of spam from amazonses.com and Amazon blocked
it quickly. From my experience, they are trustworthy and police their
outbound spam properly to trust. Otherwise you will block too much
legit email from their Simple Email Service.
https://aws.amazon.com/ses/faqs/
They have sending and bounce quotas which are going to catch most
bad actors using SES.
>the same for "whitelist_auth *@icloud.com"
Apple is also doing a good job of policing their outbound spam
from icloud.com. My logs show good reputation of the IPs.
senderscore.org report for 17.164.24.103 has a 98 out of 100
as a very high sender which is excellent.
Everyone doesn't have to whitelist_auth the same senders. I only
wanted to show that this is a valid way to reduce false positives
for transient things like Google IPs in SORBS RBL.
>[root@mail-gw:~]$ cat maillog | grep
>01000157007004fc-dd484ffc-155c-48dc-8a7d-b9fbc51b7094-000000@email.amazonses.com
>Sep 6 18:58:47 mail-gw postfix/cleanup[5554]: 3sTCVH11mDz9bQ:
>message-id=<01...@email.amazonses.com>
>Sep 6 18:58:52 mail-gw spamd[1086]: spamd: result: Y 14 -
>BAYES_99,BAYES_999,BOGOFILTER_SPAM,CUST_DNSBL_19_SPAMCANN,CUST_DNSWL_5_ORG_N,CUST_DNSWL_8_TL_N,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_D>OMAINS,HTML_MESSAGE,RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,RCVD_IN_MSPIKE_H2,RP_MATCHES_RCVD,SPF_PASS,T_OBFU_ATTACH_MISSP,URIBL_LOC>AL
>scantime=5.0,size=13908,user=sa-milt,uid=189,required_score=5.5,rhost=localhost,raddr=127.0.0.1,rport=/run/spamassassin/spamassassin.sock,mid=<01...@email.amazonses.com>,bayes=1.000000,autolearn=disabled,shortcircuit=no
Did you check the envelope-from address of that message? Those are
message IDs which wouldn't necessarily match the envelope-from
used by whitelist_auth. I don't see an IP address either to check the
source so that email could have been forwarded. I would need to see
the full headers and the message body since it did hit so many rules
and high Bayes.
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by "lists@rhsoft.net" <li...@rhsoft.net>.
Am 12.09.2016 um 15:37 schrieb David Jones:
>>Has RCVD_IN_SORBS_WEB been considered for adjustment as well? It's
>>hitting a lot more ham than spam here, including mail from facebook.
>
> You should be safely whitelisting any major senders like Facebook at
> the MTA level and in SA:
>
> whitelist_auth *@amazonses.com
for sure *not* since that would whitelist anything hosted on the amazon
cloud instances which is *not* amazon stuff itself
don't confuse major good senders with hosted crap of endcustomers
@amazonses.com != @amazon.com
the same for "whitelist_auth *@icloud.com"
[root@mail-gw:~]$ cat maillog | grep
01000157007004fc-dd484ffc-155c-48dc-8a7d-b9fbc51b7094-000000@email.amazonses.com
Sep 6 18:58:47 mail-gw postfix/cleanup[5554]: 3sTCVH11mDz9bQ:
message-id=<01...@email.amazonses.com>
Sep 6 18:58:52 mail-gw spamd[1086]: spamd: result: Y 14 -
BAYES_99,BAYES_999,BOGOFILTER_SPAM,CUST_DNSBL_19_SPAMCANN,CUST_DNSWL_5_ORG_N,CUST_DNSWL_8_TL_N,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,RCVD_IN_MSPIKE_H2,RP_MATCHES_RCVD,SPF_PASS,T_OBFU_ATTACH_MISSP,URIBL_LOCAL
scantime=5.0,size=13908,user=sa-milt,uid=189,required_score=5.5,rhost=localhost,raddr=127.0.0.1,rport=/run/spamassassin/spamassassin.sock,mid=<01...@email.amazonses.com>,bayes=1.000000,autolearn=disabled,shortcircuit=no
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by Rob McEwen <ro...@invaluement.com>.
On 9/12/2016 9:37 AM, David Jones wrote:
> The majority of the junk can be blocked with zen.spamhaus.org and
> sip.invaluement.com RBLs. Every small mail filtering platform should
> use zen.spamhaus.org as long as they are under the free usage limit.
> The sip.invaluement.com is a private RBL but very reasonably priced
> and is a great complement to zen.spamhaus.org. The major senders
> should not be listed in these 2 major RBLs so they fit right in with the
> 6 items above.
David,
Thanks so much for the recommendation!
But I just to make sure that everyone knows that throwing
sip.invaluement.com into their spam filter is not going to help anyone
as (1) it require a subscription to work -AND- (2) we now use DIFFERENT
host names. In fact, eventually, no invaluement users will be using that
host name (they'll eventually all be ported over to other alternate host
names)... and then sip.invaluement.com will be wildcarded to "list the
world"... eventually... (I'll make plenty of announcements before that
time comes). Nevertheless, there will be an unfortunately group of users
who threw sip.invaluement.com in their filter (unlicensed)... and will
leave it there even though they've never gotten a single "hit" from
their mis-configuration, and then they'll have a very bad day when that
time comes.
But, again, thanks for the mention! Perhaps, next time just say
"invaluement".
--
Rob McEwen
invaluement.com
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by David Jones <dj...@ena.com>.
>From: Alex <my...@gmail.com>
>Sent: Sunday, September 11, 2016 4:10 PM
>To: SA Mailing list
>Subject: Re: RCVD_IN_SORBS_SPAM and google IPs
>Hi,
>> COMMIT/trunk/rules/50_scores.cf
>>
>> Committed revision 1760066.
>>
>> score RCVD_IN_SORBS_SPAM 0 0.5 0 0.5
>>
>> should show up after next SA update
>Has RCVD_IN_SORBS_WEB been considered for adjustment as well? It's
>hitting a lot more ham than spam here, including mail from facebook.
You should be safely whitelisting any major senders like Facebook at
the MTA level and in SA:
whitelist_auth *@facebookmail.com
whitelist_auth *@sendgrid.net
whitelist_auth *@amazon.com
whitelist_auth *@amazonses.com
whitelist_auth *@icloud.com
whitelist_auth *@geicomail.com
whitelist_auth *@linkedin.com
There is a top tier of mail senders that should be whitelisted like this:
1. They are responsible senders with a good reputation
2. They take abuse reports seriously and will block bad senders
3. They don't have end user mailboxes that can be compromised
4. They have valid opt-out links and processes to unsubscribe
5. They are very large and send high volumes of mail so safe whitelisting
lowers the SA processing
6. 99% of the time they are going to score under the SA threshold anyway
I have a script that runs weekly to find these types of trusted senders
from my mail logs and adds them to my whitelist_auth file based on
certain criteria derived from the 6 items above. I have found patterns
that are very reliable for the SA level when you have other MTA level
things in place like postscreen with RBL weighting, reverse DNS checks,
SMTP HELO checks, etc.
Enable the Shortcircuit plugin which can also help major trusted
senders pass through:
shortcircuit USER_IN_WHITELIST on
shortcircuit USER_IN_DEF_WHITELIST on
shortcircuit USER_IN_BLACKLIST on
shortcircuit USER_IN_DKIM_WHITELIST on
shortcircuit USER_IN_DEF_DKIM_WL on
shortcircuit USER_IN_SPF_WHITELIST on
shortcircuit USER_IN_DEF_SPF_WL on
shortcircuit RCVD_IN_MSPIKE_H5 on
shortcircuit RCVD_IN_RP_CERTIFIED on
shortcircuit RCVD_IN_RP_SAFE on
shortcircuit RCVD_IN_DNSWL_HI on
shortcircuit RCVD_IN_IADB_LISTED on
shortcircuit ALL_TRUSTED off
The majority of the junk can be blocked with zen.spamhaus.org and
sip.invaluement.com RBLs. Every small mail filtering platform should
use zen.spamhaus.org as long as they are under the free usage limit.
The sip.invaluement.com is a private RBL but very reasonably priced
and is a great complement to zen.spamhaus.org. The major senders
should not be listed in these 2 major RBLs so they fit right in with the
6 items above.
A properly configured MTA should be blocking > 85 percent of the
junk so SA only has to deal with a very small percentage of email.
Even then, my SA still only has to block a very small percent of
what the MTA doesn't block. The majority of my SA traffic is
whitelisted or shortcircuited. I run MailScanner filtering about
60,000 mailboxes so SA is not integrated into the MTA. The only
complaints I get for spam by our customers is the occasional
compromised accounts which are very hard to block on zero-day
spam. They come through trusted servers that aren't listed on
any RBLs yet and they have paid sweat shops to craft the email
to get through most major mail filters.
Hope this helps,
Dave
Re: FPs on RCVD_IN_SORBS_WEB
Posted by Axb <ax...@gmail.com>.
On 03/09/2017 06:29 PM, Kevin A. McGrail wrote:
> On 3/9/2017 12:26 PM, Axb wrote:
>> On 03/09/2017 06:14 PM, Kevin A. McGrail wrote:
>>> On 3/9/2017 12:04 PM, Cedric Knight wrote:
>>>>
>>>> Compared to RCVD_IN_SORBS_SPAM, which I think Axb manually adjusted down
>>>> to 0.5 back in September, RCVD_IN_SORBS_WEB hits about a tenth as much,
>
>> atm there's a ton of hacked web servers spewing spam so I'm ok with lowering
>> the score but sugggest we try going with 1.5 and see how it goes.
>> Comments?
> +1
>
in absence of more comments I've...
COMMIT/trunk/rules/50_scores.cf
Committed revision 1786225.
score RCVD_IN_SORBS_WEB 0 1.5 0 1.5
Re: FPs on RCVD_IN_SORBS_WEB
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 3/9/2017 12:26 PM, Axb wrote:
> On 03/09/2017 06:14 PM, Kevin A. McGrail wrote:
>> On 3/9/2017 12:04 PM, Cedric Knight wrote:
>>>
>>> Compared to RCVD_IN_SORBS_SPAM, which I think Axb manually adjusted
>>> down
>>> to 0.5 back in September, RCVD_IN_SORBS_WEB hits about a tenth as much,
> atm there's a ton of hacked web servers spewing spam so I'm ok with
> lowering the score but sugggest we try going with 1.5 and see how it
> goes.
> Comments?
+1
--
*Kevin A. McGrail*
CEO
Peregrine Computer Consultants Corporation
10311 Cascade Lane
Fairfax, VA 22032
http://www.pccc.com/
703-359-9700 x50 / 800-823-8402 (Toll-Free)
703-798-0171 (wireless)
KMcGrail@PCCC.com <ma...@pccc.com>
Re: FPs on RCVD_IN_SORBS_WEB
Posted by Axb <ax...@gmail.com>.
On 03/09/2017 06:14 PM, Kevin A. McGrail wrote:
> On 3/9/2017 12:04 PM, Cedric Knight wrote:
>> Well, not based on mass checks or any advanced analysis or anything, it
>> just stops obvious Facebook etc ham being marked as spam, so working
>> much better than the previous score of 3.253.
>>
>> Compared to RCVD_IN_SORBS_SPAM, which I think Axb manually adjusted down
>> to 0.5 back in September, RCVD_IN_SORBS_WEB hits about a tenth as much,
>> but with a hit similarly being about a 25% risk of being a FP. I could
>> write some local rules to try separating out the lastexternal hits and
>> see if it eliminates some FPs, but I doubt it will. There was some
>> other experience upthread of RCVD_IN_SORBS_WEB (eg from Steve Zinski)
>> being a problem.
> If a related rule had to be adjusted down, it makes sense that this
> might have similar troubles. Axb, do you agree we should lower/cap this
> rule at 0.5 as well?
>
> If the FP rate is as high as Cedric mentions, this might be considered
> for removal but we can address that after a rule score adjustment.
>
> Regards,
> KAM
atm there's a ton of hacked web servers spewing spam so I'm ok with
lowering the score but sugggest we try going with 1.5 and see how it goes.
Comments?
Re: FPs on RCVD_IN_SORBS_WEB
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 3/9/2017 12:04 PM, Cedric Knight wrote:
> Well, not based on mass checks or any advanced analysis or anything, it
> just stops obvious Facebook etc ham being marked as spam, so working
> much better than the previous score of 3.253.
>
> Compared to RCVD_IN_SORBS_SPAM, which I think Axb manually adjusted down
> to 0.5 back in September, RCVD_IN_SORBS_WEB hits about a tenth as much,
> but with a hit similarly being about a 25% risk of being a FP. I could
> write some local rules to try separating out the lastexternal hits and
> see if it eliminates some FPs, but I doubt it will. There was some
> other experience upthread of RCVD_IN_SORBS_WEB (eg from Steve Zinski)
> being a problem.
If a related rule had to be adjusted down, it makes sense that this
might have similar troubles. Axb, do you agree we should lower/cap this
rule at 0.5 as well?
If the FP rate is as high as Cedric mentions, this might be considered
for removal but we can address that after a rule score adjustment.
Regards,
KAM
Re: FPs on RCVD_IN_SORBS_WEB
Posted by Cedric Knight <ce...@gn.apc.org>.
On 09/03/17 13:26, Kevin A. McGrail wrote:
> On 3/9/2017 8:22 AM, Cedric Knight wrote:
>> I've reduced the score on my installation to 0.5. Would this kind of
>> thing be prevented by more people contributing to the mass checks? Or
>> could it be adjusted downwards as Alex suggested?
>
> I don't know if it's a floating rule but it sounds like it needs manual
> adjustment down. How has 0.5 been working for you?
Well, not based on mass checks or any advanced analysis or anything, it
just stops obvious Facebook etc ham being marked as spam, so working
much better than the previous score of 3.253.
Compared to RCVD_IN_SORBS_SPAM, which I think Axb manually adjusted down
to 0.5 back in September, RCVD_IN_SORBS_WEB hits about a tenth as much,
but with a hit similarly being about a 25% risk of being a FP. I could
write some local rules to try separating out the lastexternal hits and
see if it eliminates some FPs, but I doubt it will. There was some
other experience upthread of RCVD_IN_SORBS_WEB (eg from Steve Zinski)
being a problem.
CK
Re: FPs on RCVD_IN_SORBS_WEB
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 3/9/2017 8:22 AM, Cedric Knight wrote:
> I've reduced the score on my installation to 0.5. Would this kind of
> thing be prevented by more people contributing to the mass checks? Or
> could it be adjusted downwards as Alex suggested?
I don't know if it's a floating rule but it sounds like it needs manual
adjustment down. How has 0.5 been working for you?
FPs on RCVD_IN_SORBS_WEB
Posted by Cedric Knight <ce...@gn.apc.org>.
On 11/09/16 22:10, Alex wrote:
>> COMMIT/trunk/rules/50_scores.cf
>>
>> Committed revision 1760066.
>>
>> score RCVD_IN_SORBS_SPAM 0 0.5 0 0.5
>>
>> should show up after next SA update
>
> Has RCVD_IN_SORBS_WEB been considered for adjustment as well? It's
> hitting a lot more ham than spam here, including mail from facebook.
Over the last four months I've seen a fair number of false positives
from RCVD_IN_SORBS_WEB, including Facebook, Google, HaveIBeenPwned and
various legit servers. A Facebook example:
145.144.220.66.dnsbl.sorbs.net. 3600 IN TXT "Exploitable Server See:
http://www.sorbs.net/lookup.shtml?66.220.144.145"
The rule scored 3.253 in November, which has fallen to 2.034 now. This
still seems high for a RBL, particularly one that does deep-parsing,
i.e. isn't -lastexternal, and hits end users (not servers) listed in the
x-originating-ip header. To be fair, it is hitting some malware and
carder spam too, but not much that would otherwise be missed. The list
is described as:
web.dnsbl.sorbs.net - List of web (WWW) servers which have spammer
abusable vulnerabilities (e.g. FormMail scripts)
Note: This zone now includes non-webserver
IP addresses that have abusable vulnerabilities.
I've reduced the score on my installation to 0.5. Would this kind of
thing be prevented by more people contributing to the mass checks? Or
could it be adjusted downwards as Alex suggested?
CK
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by Alex <my...@gmail.com>.
Hi,
> COMMIT/trunk/rules/50_scores.cf
>
> Committed revision 1760066.
>
> score RCVD_IN_SORBS_SPAM 0 0.5 0 0.5
>
> should show up after next SA update
Has RCVD_IN_SORBS_WEB been considered for adjustment as well? It's
hitting a lot more ham than spam here, including mail from facebook.
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by Axb <ax...@gmail.com>.
On 09/08/2016 10:53 PM, Shane Williams wrote:
> Hey all,
>
> I'm seeing google IP ranges hit the RCVD_IN_SORBS_SPAM rule, and in
> digging deeper, I realize that there are zero hits on this rule for
> the two weeks prior to Aug. 31, and now I'm seeing it thousands of
> times per week (not just against google IPs).
>
> Was this rule added/changed/re-scored in a recent sa-update? I looked
> at ruleqa.spamassassin.org, and just at a glance notice that the rule
> doesn't seem to be in commits previous to Aug. 30, but I may totally
> be reading the site's information wrong.
>
> I've turned the score down to a tiny, but non-zero value for now,
> because it seems to be pushing legit emails close (if not over) the
> local threshold.
>
COMMIT/trunk/rules/50_scores.cf
Committed revision 1760066.
score RCVD_IN_SORBS_SPAM 0 0.5 0 0.5
should show up after next SA update
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by "lists@rhsoft.net" <li...@rhsoft.net>.
Am 08.09.2016 um 22:53 schrieb Shane Williams:
> I'm seeing google IP ranges hit the RCVD_IN_SORBS_SPAM rule, and in
> digging deeper, I realize that there are zero hits on this rule for
> the two weeks prior to Aug. 31, and now I'm seeing it thousands of
> times per week (not just against google IPs).
>
> Was this rule added/changed/re-scored in a recent sa-update?
rules are re-scoring all the time
2.397 is *way* too high because SORBS has a ton of different scorings
and to land on "spam" is not hard for large providers which *in fact*
all day long send some amount of spam sicne large freemail providers
have no way to avoid it completly
"spam.dnsbl.sorbs.net" (127.0.0.6 response) has here 3 points on
postscreen and 1.0 for SA - in both cases reject begins with 8.0
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by sh...@shanew.net.
On Thu, 8 Sep 2016, RW wrote:
> On Thu, 8 Sep 2016 15:53:00 -0500 (CDT)
> Shane Williams wrote:
>
>> Hey all,
>>
>> I'm seeing google IP ranges hit the RCVD_IN_SORBS_SPAM rule, and in
>> digging deeper, I realize that there are zero hits on this rule for
>> the two weeks prior to Aug. 31, and now I'm seeing it thousands of
>> times per week (not just against google IPs).
>>
>> Was this rule added/changed/re-scored in a recent sa-update?
>
> It was commented out for a long time because it had a delisting fee,
> but was recently re-enabled.
>
> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=2221#c16
>
Thanks for that link, as it clarifies why it just started scoring
again.
This is the first time (at least in a long time) that I've looked at
ruleqa, but it seems like
http://ruleqa.spamassassin.org/20160904-r1759058-n/RCVD_IN_SORBS_SPAM/detail
would indicate that it should be scored at zero (since its S/O is
nearly .5), but instead it's 2.399, which is a lot to add for a rule
that's been napping for the last 13 years.
Perhaps more to the root issue, I'm concerned that it looks like
listing on SORBS is based on total volume rather than percentage.
Their summary page for the IP I checked (209.85.218.48), seems to say
that there have been 28 "recent" spam entries seen from this address,
but I would imagine this is a miniscule percentage off all email sent
from that address. If that's all it takes to get listed, I'm kind of
surprised that all of google's IPs aren't listed.
--
Public key #7BBC68D9 at | Shane Williams
http://pgp.mit.edu/ | System Admin - UT CompSci
=----------------------------------+-------------------------------
All syllogisms contain three lines | shanew@shanew.net
Therefore this is not a syllogism | www.ischool.utexas.edu/~shanew
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by Pedro David Marco <pe...@yahoo.com>.
i receive tons of Ransonware from Google and MS Office365 IPs..
---PedroD
From: Bowie Bailey <Bo...@BUC.com>
To: users@spamassassin.apache.org
Sent: Friday, September 9, 2016 3:35 PM
Subject: Re: RCVD_IN_SORBS_SPAM and google IPs
On 9/9/2016 9:24 AM, lists@rhsoft.net wrote:
>
>
> Am 09.09.2016 um 15:20 schrieb Bowie Bailey:
>> On 9/8/2016 6:29 PM, RW wrote:
>>> On Thu, 8 Sep 2016 15:53:00 -0500 (CDT)
>>> Shane Williams wrote:
>>>>
>>>> I'm seeing google IP ranges hit the RCVD_IN_SORBS_SPAM rule, and in
>>>> digging deeper, I realize that there are zero hits on this rule for
>>>> the two weeks prior to Aug. 31, and now I'm seeing it thousands of
>>>> times per week (not just against google IPs).
>>>>
>>>> Was this rule added/changed/re-scored in a recent sa-update?
>>> It was commented out for a long time because it had a delisting fee,
>>> but was recently re-enabled.
>>>
>>> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=2221#c16
>>
>> Granted, my system is fairly low volume, but out of over 15,000 messages
>> scanned, I have only seen 88 hits for SORBS rules in general and no hits
>> at all for RCVD_IN_SORBS_SPAM. If there's a problem, I'm not seeing it
>
> depends just on luck
>
> * how many mails came from gmail, yahoo, gmx & friends
> * from which server did they came
>
> sorbs don't list gmail or other freemail providers as a whole, just
> the nodes which recently was absued by spammers and contacted
> honeypots or where reported repeatly
>
> you can write the exactly same message to the same RCPT from a
> freemail provider within 5 seconds and they may hit completly
> different DNSBL/DNSWL listings
True, only 550 of my messages came from gmail or yahoo. But if Shane is
seeing thousands of hits a week, I would expect to see a few --
particularly if there is any problem with the SORBS listings or the rule
definition.
I'm not trying to draw any conclusion, I'm just providing another data
point.
--
Bowie
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by Bowie Bailey <Bo...@BUC.com>.
On 9/9/2016 9:24 AM, lists@rhsoft.net wrote:
>
>
> Am 09.09.2016 um 15:20 schrieb Bowie Bailey:
>> On 9/8/2016 6:29 PM, RW wrote:
>>> On Thu, 8 Sep 2016 15:53:00 -0500 (CDT)
>>> Shane Williams wrote:
>>>>
>>>> I'm seeing google IP ranges hit the RCVD_IN_SORBS_SPAM rule, and in
>>>> digging deeper, I realize that there are zero hits on this rule for
>>>> the two weeks prior to Aug. 31, and now I'm seeing it thousands of
>>>> times per week (not just against google IPs).
>>>>
>>>> Was this rule added/changed/re-scored in a recent sa-update?
>>> It was commented out for a long time because it had a delisting fee,
>>> but was recently re-enabled.
>>>
>>> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=2221#c16
>>
>> Granted, my system is fairly low volume, but out of over 15,000 messages
>> scanned, I have only seen 88 hits for SORBS rules in general and no hits
>> at all for RCVD_IN_SORBS_SPAM. If there's a problem, I'm not seeing it
>
> depends just on luck
>
> * how many mails came from gmail, yahoo, gmx & friends
> * from which server did they came
>
> sorbs don't list gmail or other freemail providers as a whole, just
> the nodes which recently was absued by spammers and contacted
> honeypots or where reported repeatly
>
> you can write the exactly same message to the same RCPT from a
> freemail provider within 5 seconds and they may hit completly
> different DNSBL/DNSWL listings
True, only 550 of my messages came from gmail or yahoo. But if Shane is
seeing thousands of hits a week, I would expect to see a few --
particularly if there is any problem with the SORBS listings or the rule
definition.
I'm not trying to draw any conclusion, I'm just providing another data
point.
--
Bowie
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by "lists@rhsoft.net" <li...@rhsoft.net>.
Am 09.09.2016 um 15:20 schrieb Bowie Bailey:
> On 9/8/2016 6:29 PM, RW wrote:
>> On Thu, 8 Sep 2016 15:53:00 -0500 (CDT)
>> Shane Williams wrote:
>>>
>>> I'm seeing google IP ranges hit the RCVD_IN_SORBS_SPAM rule, and in
>>> digging deeper, I realize that there are zero hits on this rule for
>>> the two weeks prior to Aug. 31, and now I'm seeing it thousands of
>>> times per week (not just against google IPs).
>>>
>>> Was this rule added/changed/re-scored in a recent sa-update?
>> It was commented out for a long time because it had a delisting fee,
>> but was recently re-enabled.
>>
>> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=2221#c16
>
> Granted, my system is fairly low volume, but out of over 15,000 messages
> scanned, I have only seen 88 hits for SORBS rules in general and no hits
> at all for RCVD_IN_SORBS_SPAM. If there's a problem, I'm not seeing it
depends just on luck
* how many mails came from gmail, yahoo, gmx & friends
* from which server did they came
sorbs don't list gmail or other freemail providers as a whole, just the
nodes which recently was absued by spammers and contacted honeypots or
where reported repeatly
you can write the exactly same message to the same RCPT from a freemail
provider within 5 seconds and they may hit completly different
DNSBL/DNSWL listings
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by Bowie Bailey <Bo...@BUC.com>.
On 9/8/2016 6:29 PM, RW wrote:
> On Thu, 8 Sep 2016 15:53:00 -0500 (CDT)
> Shane Williams wrote:
>
>> Hey all,
>>
>> I'm seeing google IP ranges hit the RCVD_IN_SORBS_SPAM rule, and in
>> digging deeper, I realize that there are zero hits on this rule for
>> the two weeks prior to Aug. 31, and now I'm seeing it thousands of
>> times per week (not just against google IPs).
>>
>> Was this rule added/changed/re-scored in a recent sa-update?
> It was commented out for a long time because it had a delisting fee,
> but was recently re-enabled.
>
> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=2221#c16
Granted, my system is fairly low volume, but out of over 15,000 messages
scanned, I have only seen 88 hits for SORBS rules in general and no hits
at all for RCVD_IN_SORBS_SPAM. If there's a problem, I'm not seeing it.
--
Bowie
Re: RCVD_IN_SORBS_SPAM and google IPs
Posted by RW <rw...@googlemail.com>.
On Thu, 8 Sep 2016 15:53:00 -0500 (CDT)
Shane Williams wrote:
> Hey all,
>
> I'm seeing google IP ranges hit the RCVD_IN_SORBS_SPAM rule, and in
> digging deeper, I realize that there are zero hits on this rule for
> the two weeks prior to Aug. 31, and now I'm seeing it thousands of
> times per week (not just against google IPs).
>
> Was this rule added/changed/re-scored in a recent sa-update?
It was commented out for a long time because it had a delisting fee,
but was recently re-enabled.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=2221#c16