You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Frederic De Mees <li...@demees.net> on 2012/12/09 11:16:08 UTC
Scoring Yahoo mail from certain continents/countries ?
Dear list,
Here is the context.
The French-speaking countries receive tons of e-mails, mostly fraud
attempts, fake lotteries, originating from West-Africa and sent by Yahoomail
users.
Often those messages contain big attachments. The payload (text of the
message) is embedded in a 1MB jpeg with fake certificates of a lawyer, a
logo, or whatever.
Spamassassin misses 100% of them because:
- the sender IP (Yahoo) is genuine and has a good reputation
- the analysis of the message text shows nothing bad, as the mill!ions of
euros are in the picture attachment
- due to the message size, the analysis is skipped anyway.
If no customer of the mail server in question expect any mail from any Yahoo
user in Africa, a simple 'header_checks' Postfix directive like this will
match such messages if their sender IP starts with 41.
/^Received: from .41\..*web.*mail.*yahoo\.com via HTTP/i
I admit this is rough albeit effective. On one side, not all Africa is 41.
On the other side, I do not want to block all 41.
I would have loved to do it with SA.
This means that the line
"Received: from [ip.add.res.ss].*web.*mail.*yahoo\.com via HTTP" should be
detected and analysed.
The ip address should be extracted.
The whois of the address should be queried.
The country code of the IP address would return certain number of SA points
from a list of "Yahoousers bad countries" I would manage.
Have I dreamed ?
Frédéric
Brussels
Re: Scoring Yahoo mail from certain continents/countries ?
Posted by John Hardin <jh...@impsec.org>.
On Sun, 9 Dec 2012, Ned Slider wrote:
> On 09/12/12 10:16, Frederic De Mees wrote:
>>
>> Spamassassin misses 100% of them because:
>> - due to the message size, the analysis is skipped anyway.
>
> look at scoring __FROM_41_FREEMAIL which already combines the above with
> FREEMAIL_FROM.
...as well as increasing the size limit for SA. SA can't do anything about
messages it doesn't scan.
One of the HTML_IMAGE_ONLY_* or HTML_IMAGE_RATIO_* rules might work well
for a meta with __FROM_41_FREEMAIL, or perhaps a new rule using
eval:image_size_range() if the images are uniformly large. You'd need to
scan these emails to see what rules hit that you could potentially meta
__FROM_41_FREEMAIL with.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
The more you believe you can create heaven on earth the more
likely you are to set up guillotines in the public square to
hasten the process. -- James Lileks
-----------------------------------------------------------------------
6 days until Bill of Rights day
Re: Scoring Yahoo mail from certain continents/countries ?
Posted by RW <rw...@googlemail.com>.
On Sat, 15 Dec 2012 19:10:06 +0100
Lutz Petersen wrote:
>
> Maybe this would be something like that you want. It first checks the
> origin country of an ip address (here limited to the 'well known' bad
> ones in africa). The second step is a meta rule that first checks if
> mail comes from yahoo (both webmail as smtp-login) and then look if
> the origin comes from one of these countries. Not really checked a
> lot, but feel free to take it as an idea. Be careful with the scores,
> otherwise all mails from those countries get hits in spamassassin:
>
>
> 1) The origin country checks:
>
>
> header __ORIGIN_COUNTRY
> eval:check_rbl('origincountry-firsthop','zz.countries.nerd.dk.')
> tflags __ORIGIN_COUNTRY noautolearn
Why do this when the relaycountry plugin can do it locally, and looks
beyond the first hop?
Re: Scoring Yahoo mail from certain continents/countries ?
Posted by Lutz Petersen <lp...@shlink.de>.
Maybe this would be something like that you want. It first checks the origin country of an ip address (here limited to the 'well known' bad ones in africa). The second step is a meta rule that first checks if mail comes from yahoo (both webmail as smtp-login) and then look if the origin comes from one of these countries. Not really checked a lot, but feel free to take it as an idea. Be careful with the scores, otherwise all mails from those countries get hits in spamassassin:
1) The origin country checks:
header __ORIGIN_COUNTRY eval:check_rbl('origincountry-firsthop','zz.countries.nerd.dk.')
tflags __ORIGIN_COUNTRY noautolearn
header ORIGIN_COUNTRY_BF eval:check_rbl_sub('origincountry-firsthop', '^127\.0\.3\.86$')
describe ORIGIN_COUNTRY_BF Origin in Burkina-Faso
tflags ORIGIN_COUNTRY_BF noautolearn
score ORIGIN_COUNTRY_BF 0.001
header ORIGIN_COUNTRY_BJ eval:check_rbl_sub('origincountry-firsthop', '^127\.0\.0\.204$')
describe ORIGIN_COUNTRY_BJ Origin in Benin
tflags ORIGIN_COUNTRY_BJ noautolearn
score ORIGIN_COUNTRY_BJ 0.001
header ORIGIN_COUNTRY_CI eval:check_rbl_sub('origincountry-firsthop', '^127\.0\.1\.128$')
describe ORIGIN_COUNTRY_CI Origin in Code-d-Ivoire
tflags ORIGIN_COUNTRY_CI noautolearn
score ORIGIN_COUNTRY_CI 0.001
header ORIGIN_COUNTRY_GA eval:check_rbl_sub('origincountry-firsthop', '^127\.0\.1\.10$')
describe ORIGIN_COUNTRY_GA Origin in Gabun
tflags ORIGIN_COUNTRY_GA noautolearn
score ORIGIN_COUNTRY_GA 0.001
header ORIGIN_COUNTRY_GH eval:check_rbl_sub('origincountry-firsthop', '^127\.0\.1\.32$')
describe ORIGIN_COUNTRY_GH Origin in Ghana
tflags ORIGIN_COUNTRY_GH noautolearn
score ORIGIN_COUNTRY_GH 0.001
header ORIGIN_COUNTRY_GM eval:check_rbl_sub('origincountry-firsthop', '^127\.0\.1\.14$')
describe ORIGIN_COUNTRY_GM Origin in Gambia
tflags ORIGIN_COUNTRY_GM noautolearn
score ORIGIN_COUNTRY_GM 0.001
header ORIGIN_COUNTRY_GN eval:check_rbl_sub('origincountry-firsthop', '^127\.0\.1\.68$')
describe ORIGIN_COUNTRY_GN Origin in Guinea
tflags ORIGIN_COUNTRY_GN noautolearn
score ORIGIN_COUNTRY_GN 0.001
header ORIGIN_COUNTRY_GW eval:check_rbl_sub('origincountry-firsthop', '^127\.0\.2\.112$')
describe ORIGIN_COUNTRY_GW Origin in Guinea-Bissao
tflags ORIGIN_COUNTRY_GW noautolearn
score ORIGIN_COUNTRY_GW 0.001
header ORIGIN_COUNTRY_NG eval:check_rbl_sub('origincountry-firsthop', '^127\.0\.2\.54$')
describe ORIGIN_COUNTRY_NG Origin in Nigeria
tflags ORIGIN_COUNTRY_NG noautolearn
score ORIGIN_COUNTRY_NG 0.001
header ORIGIN_COUNTRY_TG eval:check_rbl_sub('origincountry-firsthop', '^127\.0\.3\.0$')
describe ORIGIN_COUNTRY_TG Origin in Togo
tflags ORIGIN_COUNTRY_TG noautolearn
score ORIGIN_COUNTRY_TG 0.001
2) The yahoo check combined with some bad countries:
describe MY_YAHOO_BC Yahoo BadC
header __MY_YAHOO_HTTP X-Mailer =~ /YahooMailWebService/
header __MY_YAHOO_SMTP X-Yahoo-SMTP =~ /.*/
meta MY_YAHOO_BC (( __MY_YAHOO_HTTP + __MY_YAHOO_SMTP + ORIGIN_COUNTRY_BF + ORIGIN_COUNTRY_BJ + ORIGIN_COUNTRY_CI + ORIGIN_COUNTRY_GH + ORIGIN_COUNTRY_GM + ORIGIN_COUNTRY_GN + ORIGIN_COUNTRY_GW + ORIGIN_COUNTRY_LR + ORIGIN_COUNTRY_NG + ORIGIN_COUNTRY_TG ) > 1 )
tflags MY_YAHOO_BC noautolearn
score MY_YAHOO_BC 5.001
Re: Scoring Yahoo mail from certain continents/countries ?
Posted by Lutz Petersen <lp...@shlink.de>.
> >/^Received: from .41\..*web.*mail.*yahoo\.com via HTTP/i
> >
> >I admit this is rough albeit effective. On one side, not all Africa is
> >41. On the other side, I do not want to block all 41.
It seems that the hardcore users only resists in a limited amount of
netblocks, so it could make sense not to make one general rule but some
of them, one for any of these netranges. Sometimes these (benin for example)
are really small /23 etc. Only a small part from the home networks of our
really good friends:
41.73.158.0/24 41-73-x.isghana
41.74.0.0/24 bj/no-rdns
41.79.216.0/22 bj/no-rdns
41.85.161.0/24 bj/no-rdns
41.85.162.0/23 bj/no-rdns
41.85.169.0/24 bj/no-rdns=benin-telecom-adsl-akpakpa
41.85.224.0/21 bj/no-rdns=benin-telecom-adsl-dowa
41.138.88.0/22 bj/no-rdns=etisalat
41.138.164.0/22 ng/no-rdns=visafone
41.138.168.0/21 ng/no-rdns=visafone
41.138.176.0/20 ng/no-rdns=visafone
41.139.32.0/21 gh/no-rdns=avisonet
41.202.68.0/23 adsl-41.202.x.x.aviso.ci
41.202.88.0/21 adsl-41.202.x.x.aviso.ci
41.202.116.0/23 host-41-202-x-x.afnet.net
41.203.67.0/24 ng/no-rdns
41.203.118.0/24 ng/icsl-lagos
Re: Scoring Yahoo mail from certain continents/countries ?
Posted by Ned Slider <ne...@unixmail.co.uk>.
On 09/12/12 10:16, Frederic De Mees wrote:
> Dear list,
>
> Here is the context.
> The French-speaking countries receive tons of e-mails, mostly fraud
> attempts, fake lotteries, originating from West-Africa and sent by
> Yahoomail users.
> Often those messages contain big attachments. The payload (text of the
> message) is embedded in a 1MB jpeg with fake certificates of a lawyer, a
> logo, or whatever.
>
> Spamassassin misses 100% of them because:
> - the sender IP (Yahoo) is genuine and has a good reputation
> - the analysis of the message text shows nothing bad, as the mill!ions
> of euros are in the picture attachment
> - due to the message size, the analysis is skipped anyway.
>
> If no customer of the mail server in question expect any mail from any
> Yahoo user in Africa, a simple 'header_checks' Postfix directive like
> this will match such messages if their sender IP starts with 41.
> /^Received: from .41\..*web.*mail.*yahoo\.com via HTTP/i
>
> I admit this is rough albeit effective. On one side, not all Africa is
> 41. On the other side, I do not want to block all 41.
>
> I would have loved to do it with SA.
> This means that the line
> "Received: from [ip.add.res.ss].*web.*mail.*yahoo\.com via HTTP" should
> be detected and analysed.
> The ip address should be extracted.
> The whois of the address should be queried.
> The country code of the IP address would return certain number of SA
> points from a list of "Yahoousers bad countries" I would manage.
>
> Have I dreamed ?
>
> Frédéric
> Brussels
>
There are already 2 meta rules related to mail from 41/8 subnets:
__NSL_ORIG_FROM_41
__NSL_RCVD_FROM_41
Write some meta rules combining these with a rule for mail from Yahoo!
or look at scoring __FROM_41_FREEMAIL which already combines the above
with FREEMAIL_FROM.
That should be enough to get you started.
Re: Scoring Yahoo mail from certain continents/countries ?
Posted by Dave Funk <db...@engineering.uiowa.edu>.
On Sun, 9 Dec 2012, Frederic De Mees wrote:
> Dear list,
>
> Here is the context.
> The French-speaking countries receive tons of e-mails, mostly fraud attempts,
> fake lotteries, originating from West-Africa and sent by Yahoomail users.
> Often those messages contain big attachments. The payload (text of the
> message) is embedded in a 1MB jpeg with fake certificates of a lawyer, a
> logo, or whatever.
>
> Spamassassin misses 100% of them because:
> - the sender IP (Yahoo) is genuine and has a good reputation
> - the analysis of the message text shows nothing bad, as the mill!ions of
> euros are in the picture attachment
> - due to the message size, the analysis is skipped anyway.
>
> If no customer of the mail server in question expect any mail from any Yahoo
> user in Africa, a simple 'header_checks' Postfix directive like this will
> match such messages if their sender IP starts with 41.
> /^Received: from .41\..*web.*mail.*yahoo\.com via HTTP/i
>
> I admit this is rough albeit effective. On one side, not all Africa is 41. On
> the other side, I do not want to block all 41.
>
> I would have loved to do it with SA.
> This means that the line
> "Received: from [ip.add.res.ss].*web.*mail.*yahoo\.com via HTTP" should be
> detected and analysed.
> The ip address should be extracted.
> The whois of the address should be queried.
> The country code of the IP address would return certain number of SA points
> from a list of "Yahoousers bad countries" I would manage.
Because of its size, your message didn't get processed by SA at all.
Try a test run with the max-size parameter bumped up high enough that
SA will take a crack at it. You might find that SA is already able to deal
with that garbage.
If that works then you just need to figure out how to deal with bloated
image spams. Recently there have already been a couple different threads
on this list about exactly that issue (ranging from just increase the
max-size for everything, to make special connector that truncates bloated
spams).
Until you get SA to actually process these messages, there's no point to
discussing added bells-and-whisles.
--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{
Re: Scoring Yahoo mail from certain continents/countries ?
Posted by Alexandre Boyer <bi...@gmail.com>.
I there Frederic,
I think a geoip module exists. I saw that somewhere. Just take a look
for it.
But I think this is a bad idea. You are right about the analysis, but
geoip filtring is not efficient and may lead to FPs.
Take extra care to the rules you are going to build about it. You may
also take a look at: bayes (train your filter) et AWL.
Of course, it all depends on the size of your system.
Best,
Alex, from osmosed.
Bow before me, for I am root.
On 09/12/12 05:16 AM, Frederic De Mees wrote:
> Dear list,
>
> Here is the context.
> The French-speaking countries receive tons of e-mails, mostly fraud
> attempts, fake lotteries, originating from West-Africa and sent by
> Yahoomail users.
> Often those messages contain big attachments. The payload (text of the
> message) is embedded in a 1MB jpeg with fake certificates of a lawyer,
> a logo, or whatever.
>
> Spamassassin misses 100% of them because:
> - the sender IP (Yahoo) is genuine and has a good reputation
> - the analysis of the message text shows nothing bad, as the mill!ions
> of euros are in the picture attachment
> - due to the message size, the analysis is skipped anyway.
>
> If no customer of the mail server in question expect any mail from any
> Yahoo user in Africa, a simple 'header_checks' Postfix directive like
> this will match such messages if their sender IP starts with 41.
> /^Received: from .41\..*web.*mail.*yahoo\.com via HTTP/i
>
> I admit this is rough albeit effective. On one side, not all Africa is
> 41. On the other side, I do not want to block all 41.
>
> I would have loved to do it with SA.
> This means that the line
> "Received: from [ip.add.res.ss].*web.*mail.*yahoo\.com via HTTP"
> should be detected and analysed.
> The ip address should be extracted.
> The whois of the address should be queried.
> The country code of the IP address would return certain number of SA
> points from a list of "Yahoousers bad countries" I would manage.
>
> Have I dreamed ?
>
> Frédéric
> Brussels
Re: Scoring Yahoo mail from certain continents/countries ?
Posted by Niamh Holding <ni...@fullbore.co.uk>.
Hello Frederic,
Sunday, December 9, 2012, 10:16:08 AM, you wrote:
FDM> the sender IP (Yahoo) is genuine and has a good reputation
Good reputation! Well as a very common source of spam its reputation
is one of the best.
--
Best regards,
Niamh mailto:niamh@fullbore.co.uk
Re: Scoring Yahoo mail from certain continents/countries ?
Posted by John Hardin <jh...@impsec.org>.
On Mon, 10 Dec 2012, Frederic De Mees wrote:
> Thank you all for your replies.
> Obviously I had already increased the max size.
...so what rules *are* they hitting?
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
The social contract exists so that everyone doesn't have to squat
in the dust holding a spear to protect his woman and his meat all
day every day. It does not exist so that the government can take
your spear, your meat, and your woman because it knows better what
to do with them. -- Dagny @ Ace of Spades
-----------------------------------------------------------------------
6 days until Bill of Rights day
Re: Scoring Yahoo mail from certain continents/countries ?
Posted by Frederic De Mees <li...@demees.net>.
From: "RW" <rw...@googlemail.com>
> It's pretty easy to do this kind of thing with the RelayCountry plugin.
Thank you all for your replies.
Obviously I had already increased the max size.
I will have a look at that plugin.
Frédéric
Re: Scoring Yahoo mail from certain continents/countries ?
Posted by RW <rw...@googlemail.com>.
On Sun, 9 Dec 2012 11:16:08 +0100
Frederic De Mees wrote:
> I would have loved to do it with SA.
> This means that the line
> "Received: from [ip.add.res.ss].*web.*mail.*yahoo\.com via HTTP"
> should be detected and analysed.
> The ip address should be extracted.
> The whois of the address should be queried.
> The country code of the IP address would return certain number of SA
> points from a list of "Yahoousers bad countries" I would manage.
>
> Have I dreamed ?
It's pretty easy to do this kind of thing with the RelayCountry plugin.