You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by David Jones <dj...@ena.com> on 2017/02/16 10:07:46 UTC
Filtering outbound mail
My mail filters also do a lot of outbound relaying from hundreds
of customer mail servers. Compromised accounts happen and I
have some methods for detecting most of them and block the
sender at the MTA within a few minutes to prevent my server
IPs from becoming listed on RBLs.
Customer mail servers are currently trusted by IPs on our own
network ranges and have a slight bias toward trust by being in
the trusted_networks. This allows for the proper RBL checks
of the sender IP as long as the customer mail server adds the
proper X-Originating-IP or Received: header of the client.
The goal is to be able to block most outbound spam with the
usual rules, network tests, and Bayesian scores. However,
these compromised accounts often contain zero-hour email
that score low.
A common factor for most of these emails is sending with a
high number of recipients often to FREEMAIL recipients.
Would it make sense for me to setup/manage my own custom
rules for checking the To: header or could the FreeMail plugin
be extended to add new rules like FREEMAIL_TO?
I understand that the To: header is not the same as the
RCPT TO and the MTA will split emails based on destination.
In this situation, the sending MTA is smarthosted to my
relays and these are compromised accounts on legit MTAs
where headers can be considered reliable. I do see patterns
with sorted recipients and multiple FREEMAIL recipients
that I would like to score on. Then I have a database with
this information that I run SQL queries against to determine
frequency of certain rule hits to find compromised accounts
and block them quickly.
Thanks,
Dave
Re: Filtering outbound mail
Posted by Robert Schetterer <rs...@sys4.de>.
Am 16.02.2017 um 11:07 schrieb David Jones:
> My mail filters also do a lot of outbound relaying from hundreds
> of customer mail servers. Compromised accounts happen and I
> have some methods for detecting most of them and block the
> sender at the MTA within a few minutes to prevent my server
> IPs from becoming listed on RBLs.
>
> Customer mail servers are currently trusted by IPs on our own
> network ranges and have a slight bias toward trust by being in
> the trusted_networks. This allows for the proper RBL checks
> of the sender IP as long as the customer mail server adds the
> proper X-Originating-IP or Received: header of the client.
>
> The goal is to be able to block most outbound spam with the
> usual rules, network tests, and Bayesian scores. However,
> these compromised accounts often contain zero-hour email
> that score low.
>
> A common factor for most of these emails is sending with a
> high number of recipients often to FREEMAIL recipients.
>
> Would it make sense for me to setup/manage my own custom
> rules for checking the To: header or could the FreeMail plugin
> be extended to add new rules like FREEMAIL_TO?
>
> I understand that the To: header is not the same as the
> RCPT TO and the MTA will split emails based on destination.
> In this situation, the sending MTA is smarthosted to my
> relays and these are compromised accounts on legit MTAs
> where headers can be considered reliable. I do see patterns
> with sorted recipients and multiple FREEMAIL recipients
> that I would like to score on. Then I have a database with
> this information that I run SQL queries against to determine
> frequency of certain rule hits to find compromised accounts
> and block them quickly.
>
> Thanks,
> Dave
>
clamav-milter with sanesecurity works fine and fast at outbound
but better get an intelligent milter cross outbound smtp servers
which is able to identify hacked accounts, for i.e it counts from and to
adr, if it fades from normal traffic ,action should be taken etc ,such
exists but not as freeware and for sure it must be fitted to your needs
Best Regards
MfG Robert Schetterer
--
[*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64
Schleiheimer Strae 26/MG, 80333 Mnchen
Sitz der Gesellschaft: Mnchen, Amtsgericht Mnchen: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Re: Filtering outbound mail
Posted by Christian Grunfeld <ch...@gmail.com>.
2017-02-16 11:49 GMT-03:00 David Jones <dj...@ena.com>:
>
>
> Many of the SMTP sending software that my customers
> use are not full MTAs with queuing capabilities so some email
> would be lost if I rate limited. I also have stupid mail sending
> devices like scanners/copiers that could get lumped in with
> other SMTP traffic coming out of the same IP due to NAT.
very bad....queuing is in charge of them (every seriuos MTA does it) in
case of a failure in your relay host or net or route failure mail of your
customers will be lost even in the case you do not apply rate
limiting....bad
Re: Filtering outbound mail
Posted by David Jones <dj...@ena.com>.
>From: Reindl Harald <h....@thelounge.net>
>Sent: Thursday, February 16, 2017 8:55 AM
>To: David Jones; Spamassassin List
>Subject: Re: Filtering outbound mail
Am 16.02.2017 um 15:49 schrieb David Jones:
>> From: Christian Grunfeld <ch...@gmail.com>
>> Sent: Thursday, February 16, 2017 8:29 AM
>> To: Spamassassin List
>> Subject: Re: Filtering outbound mail
>
>> Why not rate limiting? I think everyone is doing it....I do...
>
>> Cluebringer quotas can track one to one, one to many and
>> many to one (botnets) in both directions (as sender or recipients)
>
> Many of the SMTP sending software that my customers
> use are not full MTAs with queuing capabilities so some email
> would be lost if I rate limited
>then they have no relieable delivery anyways, what when on one side or
>on a router between packet loss or restarts of any network devcies are
>happening?
I agree. We are doing good just to get them to send through our mail
relays to get reliable delivery to the Internet. They understand the risk
of network issues but they don't always understand how to setup proper
mail routing.
>a simple postfix there with a queu and SASL forwarding to your server is
>strongly recommened
Most run Windows servers so I try to get them to setup an hMailServer
as a simple internal mail relay that smarthosts to my servers when I can.
We have a large number of customers all over the United States so this
would be a full time job for multiple people if we wanted to do this
correctly everywhere. I do the best I can when I learn of problems
escalated to my team.
Re: Filtering outbound mail
Posted by David Jones <dj...@ena.com>.
>From: Christian Grunfeld <ch...@gmail.com>
>Sent: Thursday, February 16, 2017 8:29 AM
>To: Spamassassin List
>Subject: Re: Filtering outbound mail
>Why not rate limiting? I think everyone is doing it....I do...
> Cluebringer quotas can track one to one, one to many and
>many to one (botnets) in both directions (as sender or recipients)
Many of the SMTP sending software that my customers
use are not full MTAs with queuing capabilities so some email
would be lost if I rate limited. I also have stupid mail sending
devices like scanners/copiers that could get lumped in with
other SMTP traffic coming out of the same IP due to NAT.
Re: Filtering outbound mail
Posted by Christian Grunfeld <ch...@gmail.com>.
Why not rate limiting? I think everyone is doing it....I do...
Cluebringer quotas can track one to one, one to many and many to one
(botnets) in both directions (as sender or recipients)
2017-02-16 11:21 GMT-03:00 David Jones <dj...@ena.com>:
> >From: Christian Grunfeld <ch...@gmail.com>
> >Sent: Thursday, February 16, 2017 7:50 AM
> >To: Spamassassin List
> >Subject: Re: Filtering outbound mail
>
> >Are you using postfix as MTA? I use cluebringer suite which
> >has a lot of functionality (spf checks, helo checks, greylist
> >and quotas)
>
> I am using Postfix and cluebringer does looks pretty slick
> so I will check into that.
>
> >Quotas are fully configurable by tracking inbound and
> >outbound trafic by ip, sasl user, etc
>
> These outbound senders are my own internal customers
> smarthosting through my mail relays so I can't do things
> like rate limiting, greylisting, SPF checks, HELO checks,
> etc. on them like I do for Internet inbound mail.
>
> For example, they may have an Exchange server that
> sends legit emails all day long. Since I am their outbound
> mail relay, I am their Internet edge server so SPF checks
> and other network checks would be performed on my
> server by the receiving Internet mail server. I have to
> detect compromised accounts and block them to
> protect the reputation of my mail server IPs (keep them
> off of RBLs and a high senderscore.org score).
>
> My compromised account detect already works pretty
> well but I am just wanting to improve it to detect a new
> scenario. The common theme is lots of email sent to
> FREEMAIL recipients that I need a rule hit for my SQL query.
Re: Filtering outbound mail
Posted by Antony Stone <An...@spamassassin.open.source.it>.
On Friday 17 Feb 2017 at 21:51, David Jones wrote:
> Not all compromised accounts these days blast out at a high rate like we
> used to see years ago.
True, but also, some still do.
> I have had a few sneaky ones recently trickle spam through to stay below
> the radar so rate-limiting is not the answer with outbound mail
It may not be *the* answer, but it's a good (and simple) addition as _part_ of
the answer.
> I was able to build a SQL query to catch the slow sending compromised
> accounts. So far it looks reliable with a sane threshold. Just waiting for
> another compromised account to see it trigger a block.
Keep us updated.
For some folks, though, a simple solution which helps with the worst offenders
(as far as spam volume, and network bandwidth, are concerned) is worth more
than effort of creating a more complicated filter.
Antony.
--
Salad is what food eats.
Please reply to the list;
please *don't* CC me.
Re: Filtering outbound mail
Posted by Alex <my...@gmail.com>.
Hi,
>> I am using Postfix and cluebringer does looks pretty slick
>> so I will check into that.
Is that policyD?
http://wiki.policyd.org/start
It looks helpful, but hasn't had any development in at least two years.
Thanks,
Alex
Re: Filtering outbound mail
Posted by "@lbutlr" <kr...@kreme.com>.
On 2017-02-17 (14:51 MST), David Jones <dj...@ena.com> wrote:
>
>> From: @lbutlr <kr...@kreme.com>
> .Sent: Friday, February 17, 2017 3:41 PM
>> To: users@spamassassin.apache.org
>> Subject: Re: Filtering outbound mail
>
>> On 2017-02-16 (07:21 MST), David Jones <dj...@ena.com> wrote:
>>>
>>>> From: Christian Grunfeld <ch...@gmail.com>
>>>> Sent: Thursday, February 16, 2017 7:50 AM
>>>> To: Spamassassin List
>>>> Subject: Re: Filtering outbound mail
>>>>
>>>> Are you using postfix as MTA? I use cluebringer suite which
>>>> has a lot of functionality (spf checks, helo checks, greylist
>>>> and quotas)
>>>
>>> I am using Postfix and cluebringer does looks pretty slick
>>> so I will check into that.
>>>
>>>> Quotas are fully configurable by tracking inbound and
>>>> outbound trafic by ip, sasl user, etc
>>>
>>> These outbound senders are my own internal customers
>>> smarthosting through my mail relays so I can't do things
>>> like rate limiting, greylisting, SPF checks, HELO checks,
>>> etc. on them like I do for Internet inbound mail.
>
>> Oh yes you can, and yes you should. At the very least a
>> sane rate-limit will catch instances where customers get
>> compromised.
>
> Not all compromised accounts these days blast out at a
> high rate like we used to see years ago. I have had a few
> sneaky ones recently trickle spam through to stay below
> the radar so rate-limiting is not the answer with outbound
> mail
I never said it was THE answer, but it most certainly is AN answer.
--
Apple broke AppleScripting signatures in Mail.app, so no random signatures.
Re: Filtering outbound mail
Posted by David Jones <dj...@ena.com>.
>From: @lbutlr <kr...@kreme.com>
.Sent: Friday, February 17, 2017 3:41 PM
>To: users@spamassassin.apache.org
>Subject: Re: Filtering outbound mail
>On 2017-02-16 (07:21 MST), David Jones <dj...@ena.com> wrote:
>>
>>> From: Christian Grunfeld <ch...@gmail.com>
>>> Sent: Thursday, February 16, 2017 7:50 AM
>>> To: Spamassassin List
>>> Subject: Re: Filtering outbound mail
>>>
>>> Are you using postfix as MTA? I use cluebringer suite which
>>> has a lot of functionality (spf checks, helo checks, greylist
>>> and quotas)
>>
>> I am using Postfix and cluebringer does looks pretty slick
>> so I will check into that.
>>
>>> Quotas are fully configurable by tracking inbound and
>>> outbound trafic by ip, sasl user, etc
>>
>> These outbound senders are my own internal customers
>> smarthosting through my mail relays so I can't do things
>> like rate limiting, greylisting, SPF checks, HELO checks,
>> etc. on them like I do for Internet inbound mail.
>Oh yes you can, and yes you should. At the very least a
>sane rate-limit will catch instances where customers get
>compromised.
Not all compromised accounts these days blast out at a
high rate like we used to see years ago. I have had a few
sneaky ones recently trickle spam through to stay below
the radar so rate-limiting is not the answer with outbound
mail
I was able to build a SQL query to catch the slow sending
compromised accounts. So far it looks reliable with a
sane threshold. Just waiting for another compromised
account to see it trigger a block.
Dave
Re: Filtering outbound mail
Posted by "@lbutlr" <kr...@kreme.com>.
On 2017-02-16 (07:21 MST), David Jones <dj...@ena.com> wrote:
>
>> From: Christian Grunfeld <ch...@gmail.com>
>> Sent: Thursday, February 16, 2017 7:50 AM
>> To: Spamassassin List
>> Subject: Re: Filtering outbound mail
>
>> Are you using postfix as MTA? I use cluebringer suite which
>> has a lot of functionality (spf checks, helo checks, greylist
>> and quotas)
>
> I am using Postfix and cluebringer does looks pretty slick
> so I will check into that.
>
>> Quotas are fully configurable by tracking inbound and
>> outbound trafic by ip, sasl user, etc
>
> These outbound senders are my own internal customers
> smarthosting through my mail relays so I can't do things
> like rate limiting, greylisting, SPF checks, HELO checks,
> etc. on them like I do for Internet inbound mail.
Oh yes you can, and yes you should. At the very least a sane rate-limit will catch instances where customers get compromised.
--
Apple broke AppleScripting signatures in Mail.app, so no random signatures.
Re: Filtering outbound mail
Posted by David Jones <dj...@ena.com>.
>From: Christian Grunfeld <ch...@gmail.com>
>Sent: Thursday, February 16, 2017 7:50 AM
>To: Spamassassin List
>Subject: Re: Filtering outbound mail
>Are you using postfix as MTA? I use cluebringer suite which
>has a lot of functionality (spf checks, helo checks, greylist
>and quotas)
I am using Postfix and cluebringer does looks pretty slick
so I will check into that.
>Quotas are fully configurable by tracking inbound and
>outbound trafic by ip, sasl user, etc
These outbound senders are my own internal customers
smarthosting through my mail relays so I can't do things
like rate limiting, greylisting, SPF checks, HELO checks,
etc. on them like I do for Internet inbound mail.
For example, they may have an Exchange server that
sends legit emails all day long. Since I am their outbound
mail relay, I am their Internet edge server so SPF checks
and other network checks would be performed on my
server by the receiving Internet mail server. I have to
detect compromised accounts and block them to
protect the reputation of my mail server IPs (keep them
off of RBLs and a high senderscore.org score).
My compromised account detect already works pretty
well but I am just wanting to improve it to detect a new
scenario. The common theme is lots of email sent to
FREEMAIL recipients that I need a rule hit for my SQL query.
Re: Filtering outbound mail
Posted by Christian Grunfeld <ch...@gmail.com>.
Are you using postfix as MTA? I use cluebringer suite which has a lot of
functionality (spf checks, helo checks, greylist and quotas)
Quotas are fully configurable by tracking inbound and outbound trafic by
ip, sasl user, etc
2017-02-16 9:44 GMT-03:00 David Jones <dj...@ena.com>:
> >From: Axb <ax...@gmail.com>
> >Sent: Thursday, February 16, 2017 4:54 AM
> >To: users@spamassassin.apache.org
> >Subject: Re: Filtering outbound mail
>
> >On 02/16/2017 11:07 AM, David Jones wrote:
> >> Would it make sense for me to setup/manage my own custom
> >> rules for checking the To: header or could the FreeMail plugin
> >> be extended to add new rules like FREEMAIL_TO?
>
> >To block outbound bursts using SA is probably the most inneficient method.
>
> >Fai2ban is probably safer / easier to manage
> >Also, look into inbound rating per sender / IP & time period.
>
> I have implemented rate limiting and very accurate RBL
> checking on inbound mail.
>
> I can't do blocking with fail2ban or rate limiting on outbound
> customer mail since not all of them setup a dedicated
> NAT IP for their servers that send email so blocking an IP
> could have multiple servers behind that NAT IP.
>
> Our primary customers are K12 education and libraries
> which have automated software that blast out emails
> to parents and patrons for school attendance, grades,
> progress reports, and book overdue reports. I have
> whitelisted these types of emails with a SHORTCIRCUIT
> rule that is excluded from the compromised account
> detection.
>
> I guess I will setup/maintain my own FREEMAIL_TO
> rules but I thought that others would also have the
> same need. Maybe not. Seemed logical to extend
> the FreeMail plugin to add a few new rules.
>
> Dave
>
>
Re: Filtering outbound mail
Posted by David Jones <dj...@ena.com>.
>From: Axb <ax...@gmail.com>
>Sent: Thursday, February 16, 2017 4:54 AM
>To: users@spamassassin.apache.org
>Subject: Re: Filtering outbound mail
>On 02/16/2017 11:07 AM, David Jones wrote:
>> Would it make sense for me to setup/manage my own custom
>> rules for checking the To: header or could the FreeMail plugin
>> be extended to add new rules like FREEMAIL_TO?
>To block outbound bursts using SA is probably the most inneficient method.
>Fai2ban is probably safer / easier to manage
>Also, look into inbound rating per sender / IP & time period.
I have implemented rate limiting and very accurate RBL
checking on inbound mail.
I can't do blocking with fail2ban or rate limiting on outbound
customer mail since not all of them setup a dedicated
NAT IP for their servers that send email so blocking an IP
could have multiple servers behind that NAT IP.
Our primary customers are K12 education and libraries
which have automated software that blast out emails
to parents and patrons for school attendance, grades,
progress reports, and book overdue reports. I have
whitelisted these types of emails with a SHORTCIRCUIT
rule that is excluded from the compromised account
detection.
I guess I will setup/maintain my own FREEMAIL_TO
rules but I thought that others would also have the
same need. Maybe not. Seemed logical to extend
the FreeMail plugin to add a few new rules.
Dave
Re: Filtering outbound mail
Posted by Axb <ax...@gmail.com>.
On 02/16/2017 11:07 AM, David Jones wrote:
> Would it make sense for me to setup/manage my own custom
> rules for checking the To: header or could the FreeMail plugin
> be extended to add new rules like FREEMAIL_TO?
To block outbound bursts using SA is probably the most inneficient method.
Fai2ban is probably safer / easier to manage
Also, look into inbound rating per sender / IP & time period.
Re: Filtering outbound mail
Posted by David Jones <dj...@ena.com>.
>From: Dianne Skoll <df...@roaringpenguin.com>
>Sent: Thursday, February 16, 2017 8:30 AM
>To: users@spamassassin.apache.org
>Subject: Re: Filtering outbound mail
>On Thu, 16 Feb 2017 10:07:46 +0000
>David Jones <dj...@ena.com> wrote:
>> Would it make sense for me to setup/manage my own custom
>> rules for checking the To: header or could the FreeMail plugin
>> be extended to add new rules like FREEMAIL_TO?
>The To: header may not contain useful information. I don't think
>the usual spam-filtering techniques are appropriate for blocking
>internal abusers; I think you want to apply some sort of rate-limiting
>that blocks senders (possibly domains and IP addresses) that exceed some
>number of recipients per hour.
I understand that BCC'ing makes the To: header not completely
reliable but I would like to be able to catch it when it's there.
>It's not trivial to set this up, unfortunately.
I agree. Thanks for the hint. I think I already have this
information in my MailWatch database and just need
to come up with a query to count the recipients per
envelope-from over a period of time.
Re: Filtering outbound mail
Posted by Dianne Skoll <df...@roaringpenguin.com>.
On Thu, 16 Feb 2017 10:07:46 +0000
David Jones <dj...@ena.com> wrote:
> Would it make sense for me to setup/manage my own custom
> rules for checking the To: header or could the FreeMail plugin
> be extended to add new rules like FREEMAIL_TO?
The To: header may not contain useful information. I don't think
the usual spam-filtering techniques are appropriate for blocking
internal abusers; I think you want to apply some sort of rate-limiting
that blocks senders (possibly domains and IP addresses) that exceed some
number of recipients per hour.
It's not trivial to set this up, unfortunately.
Regards,
Dianne.