You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Charles Gregory <cg...@hwcn.org> on 2010/03/29 19:09:26 UTC
ATTN DEVELOPERS: Mega-Spam
Literally, Mega-Spam. I just got a spam with 1MB of images.
My suggestion has been made before, but I would like to ask that it now
be taken a bit more seriously. SA needs an option to allow efficient
'partial' scanning of large e-mails, so that, for example, we can
peform all the valuable header checks, and maybe even scan for URIBL hits
within the first few hundred K of the body?
Is it possible (and easy!) to set a flag that tells SA to stop testing
aganist the body when it reaches a certain byte count.... Or perhaps, if
I understand the docs correctly, most rules only trigger on textual
message parts anyway, so by simply disabling 'full' rules and possbily
'rawbody', we could get the desired result without too much of a
processing hit?
- C
Re: ATTN DEVELOPERS: Mega-Spam
Posted by Michael Scheidell <sc...@secnap.net>.
On 3/29/10 1:09 PM, Charles Gregory wrote:
>
> Literally, Mega-Spam. I just got a spam with 1MB of images.
>
> My suggestion has been made before, but I would like to ask that it
> now be taken a bit more seriously. SA needs an option to allow efficient
> 'partial' scanning of large e-mails, so that, for example, we can
> peform all the valuable header checks, and maybe even scan for URIBL
> hits within the first few hundred K of the body?
could, will and does mess up dkim checks and language checks.
That said, amavisd-new has a switch to do this already, and for the very
same reason.
(yes, it costs the scumbags nothing to have aunt martha and her zombot
send out 600MM 1MB spams)
--
Michael Scheidell, CTO
Phone: 561-999-5000, x 1259
> *| *SECNAP Network Security Corporation
* Certified SNORT Integrator
* 2008-9 Hot Company Award Winner, World Executive Alliance
* Five-Star Partner Program 2009, VARBusiness
* Best Anti-Spam Product 2008, Network Products Guide
* King of Spam Filters, SC Magazine 2008
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________
Re: ATTN DEVELOPERS: Mega-Spam
Posted by Martin Gregorie <ma...@gregorie.org>.
On Mon, 2010-03-29 at 23:01 +0200, Mathias Homann wrote:
> I think it has, I get about 2-5 mega spams per day by now.
> and I can't do greylisting because I have to fetchmail from a central
> mail server at my hoster that is not under my direct control.
> And no, moving from a vhost to a root server just to be able to
> greylist is not an option. 5 euro per month versus 50 euro per
> month...
>
Can you persuade your hosting site to implement grey-listing?
My ISP implemented grey-listing over a year ago. When they did, my
overall spam rate immediately dropped from 80% of my mail stream to
under 10%. Currently spam is running at less than 5%. As a result my SA
subsystem is mostly trapping spam sent over less-common channels, e.g.
mailing lists and an ISP-provided address I no longer use.
Having to handle a stream of large spam messages can't be improving the
throughput and disk usage of your hosting site's mail server either, so
its worth pointing that out to them. They may be more amenable to
introducing grey-listing than you realise.
Martin
Re: ATTN DEVELOPERS: Mega-Spam
Posted by Mathias Homann <ad...@eregion.de>.
Am Montag 29 März 2010 schrieb Karsten Bräckelmann:
> On Mon, 2010-03-29 at 16:23 -0400, Brent Kennedy wrote:
> > Wow, I knew this was coming at some point. I just figured it was
> > too expensive.
>
> You did read the entire thread, right? :) There's nothing new
> about this. Moreover, this still is a rare occurrence. Note even
> Charles, who started this thread, claims to have received *one*
> such spam. And it appears to be his first. ;)
>
> Now, if this starts to become a more general pattern...
I think it has, I get about 2-5 mega spams per day by now.
and I can't do greylisting because I have to fetchmail from a central
mail server at my hoster that is not under my direct control.
And no, moving from a vhost to a root server just to be able to
greylist is not an option. 5 euro per month versus 50 euro per
month...
bye,
MH
Re: ATTN DEVELOPERS: Mega-Spam
Posted by Mark Martinec <Ma...@ijs.si>.
> We've got plenty of time, but I suggest not waiting until it becomes a
> big problem before desperately rushing to fix it.... :)
Depends on how one defines where a problem starts to become 'big'.
For me the problem of large messages was big enough early last year so
that I had to implement a solution for it in amavisd-new 2.6.3 - with
a corresponding support in the SpamAssassin 3.3.0 library (most of it
in its DKIM plugin, as mentioned by Michael).
From 2.6.3 release notes (April 22, 2009):
- large messages beyond $sa_mail_body_size_limit are now partially passed
to SpamAssassin and other spam scanners for checking: a copy passed to
a spam scanner is truncated near or slightly past the indicated limit.
Large messages are no longer given an almost free passage through spam
checks.
Note that message truncation can invalidate a DKIM or DK signature.
If using (non-default) SpamAssassin rules to assign score points to mail
with no valid signatures from authors which are expected to always provide
a valid signature, the message truncation can cause false positives on
these rules. As a workaround, to a truncated message passed to spam
scanners, amavisd inserts a header field:
X-Amavis-MessageSize: mmmmm, TRUNCATED to nnnnn
which can be captured by SpamAssassin rules, e.g.:
header __TRUNCATED X-Amavis-MessageSize =~ m{\A[^\n]*TRUNCATED}m
and used in rules like NOTVALID_EBAY to prevent them from triggering.
Starting with version 3.3.0 of SpamAssassin, its DKIM plugin understands
the issue and receives undamaged DKIM signature objects directly from
amavisd, so the above workaround is not needed. Also, a hit on a __TRUNCATED
rule is automatically generated (explicit header rule is not necessary),
just in case it might be useful for some purpose.
Just did a grep on our log, seeking out large spam messages (beyond 420 kB)
and print their sizes. Below is the complete list for March 2010. Seems we
are getting about 4.5 large spam messages daily on the average (out of
about 75k messages daily). A lot? Depends on one's point of view.
Mar 1 score: 12.7, size: 533 kB
Mar 1 score: 23.9, size: 435 kB
Mar 1 score: 16.4, size: 533 kB
Mar 1 score: 7.4, size: 490 kB
Mar 1 score: 9.1, size: 490 kB
Mar 1 score: 19.6, size: 721 kB
Mar 2 score: 15.1, size: 1132 kB
Mar 2 score: 16.9, size: 643 kB
Mar 2 score: 16.9, size: 643 kB
Mar 2 score: 7.3, size: 587 kB
Mar 2 score: 21.9, size: 721 kB
Mar 2 score: 21.6, size: 527 kB
Mar 2 score: 24.8, size: 436 kB
Mar 2 score: 20.5, size: 527 kB
Mar 2 score: 20.6, size: 528 kB
Mar 3 score: 23.6, size: 435 kB
Mar 3 score: 30.4, size: 543 kB
Mar 3 score: 30.2, size: 543 kB
Mar 3 score: 30.2, size: 543 kB
Mar 3 score: 31.5, size: 543 kB
Mar 3 score: 18.3, size: 1132 kB
Mar 3 score: 31.5, size: 543 kB
Mar 4 score: 31.5, size: 543 kB
Mar 4 score: 32.9, size: 543 kB
Mar 4 score: 10.3, size: 719 kB
Mar 4 score: 10.3, size: 719 kB
Mar 4 score: 10.1, size: 719 kB
Mar 4 score: 10.0, size: 720 kB
Mar 4 score: 10.2, size: 719 kB
Mar 5 score: 16.0, size: 1132 kB
Mar 5 score: 24.2, size: 513 kB
Mar 5 score: 25.9, size: 513 kB
Mar 6 score: 9.9, size: 719 kB
Mar 7 score: 29.6, size: 699 kB
Mar 7 score: 12.3, size: 682 kB
Mar 7 score: 10.2, size: 433 kB
Mar 7 score: 10.2, size: 433 kB
Mar 7 score: 38.0, size: 543 kB
Mar 7 score: 38.0, size: 543 kB
Mar 7 score: 38.0, size: 543 kB
Mar 7 score: 7.5, size: 1787 kB
Mar 7 score: 18.1, size: 643 kB
Mar 7 score: 38.0, size: 543 kB
Mar 7 score: 38.0, size: 543 kB
Mar 8 score: 38.0, size: 543 kB
Mar 8 score: 18.2, size: 643 kB
Mar 8 score: 7.1, size: 1050 kB
Mar 8 score: 18.0, size: 1132 kB
Mar 8 score: 25.7, size: 501 kB
Mar 9 score: 30.6, size: 813 kB
Mar 9 score: 13.7, size: 779 kB
Mar 10 score: 36.9, size: 470 kB
Mar 10 score: 29.3, size: 1407 kB
Mar 10 score: 13.5, size: 910 kB
Mar 11 score: 8.0, size: 812 kB
Mar 11 score: 8.4, size: 821 kB
Mar 11 score: 25.5, size: 435 kB
Mar 12 score: 8.8, size: 857 kB
Mar 12 score: 32.5, size: 543 kB
Mar 12 score: 32.5, size: 543 kB
Mar 12 score: 32.5, size: 543 kB
Mar 12 score: 32.5, size: 543 kB
Mar 12 score: 32.5, size: 543 kB
Mar 12 score: 32.5, size: 543 kB
Mar 12 score: 32.5, size: 543 kB
Mar 12 score: 32.5, size: 543 kB
Mar 12 score: 32.5, size: 543 kB
Mar 13 score: 32.5, size: 543 kB
Mar 13 score: 7.8, size: 732 kB
Mar 13 score: 7.9, size: 926 kB
Mar 13 score: 7.7, size: 732 kB
Mar 13 score: 19.9, size: 513 kB
Mar 14 score: 8.7, size: 821 kB
Mar 14 score: 8.8, size: 821 kB
Mar 14 score: 8.8, size: 821 kB
Mar 14 score: 8.7, size: 821 kB
Mar 14 score: 8.8, size: 821 kB
Mar 14 score: 8.7, size: 821 kB
Mar 14 score: 10.9, size: 564 kB
Mar 15 score: 22.6, size: 436 kB
Mar 15 score: 12.5, size: 558 kB
Mar 15 score: 8.3, size: 818 kB
Mar 15 score: 7.6, size: 818 kB
Mar 16 score: 28.5, size: 501 kB
Mar 16 score: 7.1, size: 925 kB
Mar 16 score: 11.3, size: 2484 kB
Mar 16 score: 9.1, size: 1964 kB
Mar 17 score: 7.0, size: 771 kB
Mar 17 score: 12.7, size: 539 kB
Mar 18 score: 23.8, size: 436 kB
Mar 19 score: 18.4, size: 499 kB
Mar 19 score: 30.3, size: 543 kB
Mar 19 score: 23.6, size: 435 kB
Mar 19 score: 23.0, size: 436 kB
Mar 19 score: 34.5, size: 543 kB
Mar 19 score: 34.6, size: 543 kB
Mar 19 score: 22.5, size: 513 kB
Mar 19 score: 23.9, size: 436 kB
Mar 19 score: 22.1, size: 512 kB
Mar 19 score: 22.5, size: 513 kB
Mar 19 score: 22.5, size: 513 kB
Mar 19 score: 34.6, size: 543 kB
Mar 19 score: 27.3, size: 513 kB
Mar 19 score: 7.2, size: 884 kB
Mar 19 score: 23.5, size: 436 kB
Mar 19 score: 28.1, size: 513 kB
Mar 19 score: 23.5, size: 436 kB
Mar 19 score: 28.1, size: 513 kB
Mar 19 score: 34.6, size: 543 kB
Mar 20 score: 34.7, size: 543 kB
Mar 20 score: 35.4, size: 543 kB
Mar 21 score: 10.8, size: 2925 kB
Mar 21 score: 37.0, size: 543 kB
Mar 21 score: 37.0, size: 543 kB
Mar 21 score: 20.6, size: 513 kB
Mar 22 score: 40.7, size: 470 kB
Mar 24 score: 29.1, size: 749 kB
Mar 25 score: 8.5, size: 625 kB
Mar 26 score: 22.8, size: 523 kB
Mar 26 score: 17.6, size: 1979 kB
Mar 27 score: 9.0, size: 577 kB
Mar 27 score: 14.8, size: 472 kB
Mar 28 score: 7.1, size: 542 kB
Mar 28 score: 9.4, size: 440 kB
Mar 29 score: 11.0, size: 578 kB
Mar 29 score: 11.0, size: 578 kB
Mar 29 score: 7.0, size: 7652 kB
Mar 29 score: 18.7, size: 691 kB
Mar 29 score: 28.8, size: 497 kB
Mar 29 score: 32.5, size: 497 kB
Mark
Re: Mega-Spam
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Tue, 2010-03-30 at 09:38 -0400, Charles Gregory wrote:
> (Subject line changed to remove the 'flag' to developers)
>
> On Mon, 29 Mar 2010, Karsten Bräckelmann wrote:
> > ...... But then again, this is a topic for the dev
> > list [1] to start a discussion, not here.
>
> Uh, no, I'm not a developer. And the description of that list
> specifically says...
>
> "For those involved in the development effort to discuss their work on
> the project. Unless you are working on a patch to SpamAssassin, this
> is probably not a list you need to use. If you're not already on the
"probably not ... need". You are allowed to go there, though. :)
> THIS is the list for users to ask questions *and* make suggestions, and
> has been used this way many times in the past. And the list description of
> this list says (and personal experience has proven) that developers
> monitor this list "as time permits".
Most of us devs do, the active ones at least. Some even try to help
solve user's issues and confusion.
> So with respect, please stop telling people to clutter up a working list
> with (possibly) dumb ideas. And this one was mine, so I can call it dumb
> if I want to. I'm thinking it's not, but open to the idea it may be. :)
I am and will continue to tell people who believe their idea is worth-
while enough to call for the devs, to go where they are. Just like I
sometimes try to sanitize and moderate this list with respect to OT
discussions going out of hand.
> > [1] Also note your very own Subject.
>
> Intended to grab the attention of time-constrained developers, though
> honestly I am regretting it now, because I don't like having that 'all
> caps' flag attached to a discussion. I've removed it from this post....
Exactly. If you want the attention of time-constrained devs, you know
where to go.
guenther
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Mega-Spam
Posted by Charles Gregory <cg...@hwcn.org>.
(Subject line changed to remove the 'flag' to developers)
On Mon, 29 Mar 2010, Karsten Bräckelmann wrote:
> ...... But then again, this is a topic for the dev
> list [1] to start a discussion, not here.
Uh, no, I'm not a developer. And the description of that list
specifically says...
"For those involved in the development effort to discuss their work on
the project. Unless you are working on a patch to SpamAssassin, this
is probably not a list you need to use. If you're not already on the
general users list, you should probably go there first..."
THIS is the list for users to ask questions *and* make suggestions, and
has been used this way many times in the past. And the list description of
this list says (and personal experience has proven) that developers
monitor this list "as time permits".
So with respect, please stop telling people to clutter up a working list
with (possibly) dumb ideas. And this one was mine, so I can call it dumb
if I want to. I'm thinking it's not, but open to the idea it may be. :)
As a side note, I had not intended to open a discussion, but merely drop
my suggestion in the 'suggestion box', but I am glad I posted here,
because the discusion among users has clarified the extent of the
problem. :)
> [1] Also note your very own Subject.
Intended to grab the attention of time-constrained developers, though
honestly I am regretting it now, because I don't like having that 'all
caps' flag attached to a discussion. I've removed it from this post....
- Charles
Re: ATTN DEVELOPERS: Mega-Spam
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Mon, 2010-03-29 at 16:57 -0400, Charles Gregory wrote:
> The spams I've seen so far look more 'amateur' than 'pro'. Easily tracable
> IP's. Blacklistable domains. I'm just throwing my idea into the queue now
> so that it can be smoothly integrated with a future release. We've got
> plenty of time, but I suggest not waiting until it becomes a big problem
> before desperately rushing to fix it.... :)
Agreed on the latter part. But then again, this is a topic for the dev
list [1] to start a discussion, not here.
guenther
[1] Also note your very own Subject.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: ATTN DEVELOPERS: Mega-Spam
Posted by Charles Gregory <cg...@hwcn.org>.
On Mon, 29 Mar 2010, Karsten Bräckelmann wrote:
> You did read the entire thread, right? :) There's nothing new about
> this. Moreover, this still is a rare occurrence. Note even Charles, who
> started this thread, claims to have received *one* such spam. And it
> appears to be his first. ;)
Last September the number of spams exceeding 256KB became frequent enough
that I bumped up my limit. Now I'm starting to see spams past the new
limit (400KB). But when they jump up to 1MB, maybe it's time for a
different solution, and maybe regain some of system efficiency by adding
the suggested mechanism to SA and only doing significant body scans on
messages less than 256KN again.... :)
> Now, if this starts to become a more general pattern...
The spams I've seen so far look more 'amateur' than 'pro'. Easily tracable
IP's. Blacklistable domains. I'm just throwing my idea into the queue now
so that it can be smoothly integrated with a future release. We've got
plenty of time, but I suggest not waiting until it becomes a big problem
before desperately rushing to fix it.... :)
My 0.02 dollars....
- C
RE: ATTN DEVELOPERS: Mega-Spam
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Mon, 2010-03-29 at 16:23 -0400, Brent Kennedy wrote:
> Wow, I knew this was coming at some point. I just figured it was too
> expensive.
You did read the entire thread, right? :) There's nothing new about
this. Moreover, this still is a rare occurrence. Note even Charles, who
started this thread, claims to have received *one* such spam. And it
appears to be his first. ;)
Now, if this starts to become a more general pattern...
guenther
> -----Original Message-----
> From: Charles Gregory [mailto:cgregory@hwcn.org]
>
> Literally, Mega-Spam. I just got a spam with 1MB of images.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
RE: ATTN DEVELOPERS: Mega-Spam
Posted by John Hardin <jh...@impsec.org>.
On Mon, 29 Mar 2010, Brent Kennedy wrote:
> Ya know, this got me thinking. Wonder if I could create a VM with all the
> settings and a script to customize the setup. Then organizations could just
> deploy the VM. Sort of an all in one deployment. Just update the VM
> template every now and then. Ahh but the learning db might be an issue....
> oh well just a thought.
A second VM hosting the bayes DB on MYSQL or Postgres. That way you can
drop-in upgrade the SA vm without destabilizing the bayes DB VM.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Our government wants to do everything it can "for the children,"
except sparing them crushing tax burdens.
-----------------------------------------------------------------------
3 days until April Fools' day
Re: ATTN DEVELOPERS: Mega-Spam
Posted by Kai Schaetzl <ma...@conactive.com>.
Jonas Eckerman wrote on Tue, 30 Mar 2010 11:52:43 +0200:
> I suspect that If the botnets were short on bandwidth and computer
> power, the programmers would have fix those issues a long time ago. And
> the simple fact that they still haven't adapted to greylisting indicates
> that it's impact is not (so far) big enough to care about.
And that's good for all of us using greylisting :-)
Kai
--
Get your web at Conactive Internet Services: http://www.conactive.com
Re: [SA] on greylisting...
Posted by Adam Katz <an...@khopis.com>.
Adam Katz wrote:
> matching p0f's results with the (perl-compatible) regular expression
> /Windows (?:XP|2000(?!SP4)|Vista)/
> will safely block only desktops.
Oops, that lost a space after the "2000" part. That should read:
/Windows (?:XP|2000 (?!SP4)|Vista)/
Also note that p0f is currently incapable of detecting newer items
like Vista, 2008, and 7. I've included Vista it in my regex and 2008
in my milter-greylist exceptions for completeness in case of updates.
Re: on greylisting...
Posted by Jonas Eckerman <jo...@frukt.org>.
On 2010-04-01 19:06, Adam Katz wrote:
> For what it's worth, I reconfigured my greylisting relay from a
> blanket delay to delaying only spamcop neighbors, anything that hits a
> DNSBL, and any Windows *desktop* (using p0f).
I once tried that, had had to refrain from it. The groupware system
FirstClass installed on Windows NT+ (of different flavors, including
"desktop" OSes) machines is (or was) popular with swedish disability
NGOs, and beeing an NGO for deafblind people, we need to be able to
communicate those systems.
I probably should analyze our current mail stream to see if we still get
lots of mail from FC systems, and what OSes those seem to be running on
nowadays.
(The fact that admins of above mentioned FirstClass systems tended to
configure outgoing SMTP in "odd" ways also amde m putin some
country/domainbased exemptions...)
> If I recall correctly, Jonas's implementation also uses p0f and could
> therefore benefit from my analysis.
Yes, my implementation can use p0f. It uses a list of tests that are
checked in order to decide wether a sending system sould be handled by
the grylist or not.
I'm currently using tests for OS (p0f), DNS black- and white-lists,
RDNS, MX, SPF, country (GeoIP), sender domain, local spam/ham history
and local otgoing hitory to make that desicion.
> p0f's results with the (perl-compatible) regular expression
> /Windows (?:XP|2000(?!SP4)|Vista)/
> will safely block only desktops.
Interesting. I hope I'll have time to check that against or logs. It
would nice to have windows desktops greylisted while still beeing able
to exempt windows mail and groupware systems.
Regards
/Jonas
--
Jonas Eckerman
Fruktträdet & Förbundet Sveriges Dövblinda
http://www.fsdb.org/
http://www.frukt.org/
http://whatever.frukt.org/
Re: on greylisting...
Posted by Adam Katz <an...@khopis.com>.
Ned Slider wrote:
> Some implementations, postgrey for example, keep a whitelist of
> servers known to retry, so once a small number of mails (3 by
> default iirc) have been successfully delivered from a given server
> (or servers in the same /24 subnet), it is auto-whitelisted on the
> basis that there is little point continually greylisting servers
> that you *know* will retry anyway.
>
> This approach works extremely well and after a few weeks normal
> usage very few legitimate mails are delayed by greylisting.
My grey time is 35 days, which is effectively the same thing. By
greylisting only Windows desktops, I can ensure emails sent during a
conference call are received without delay, during a support call,
etc. (though the support address is configured to bypass greylisting
during the work day). A server's first connection matters!
That said, it would be nice to revoke the grey time for a server that
didn't come back. That might even get me to make the grey time
bigger, though the larger the database, the longer it takes to search
(which must be a larger problem with postgrey with all that
non-expiring data...).
Re: on greylisting...
Posted by Adam Katz <an...@khopis.com>.
Matt wrote:
> I wrote a perl script that whitelists any servers from greylisting for
> 6 months that send a message that scores less then 1 by spamassassin.
> If it later sends a message that scores greater then 5 it is removed
> from the whitelist. Works great. After having a few months to learn
> almost no legitimate servers are greylisted.
Milter-greylist can do that too. I haven't had the time yet, but I'll
be eventually moving my relays to use milter-greylist for greylisting
AND spamassassin, which would also let me mix and match.
That could look something like this in your greylist.conf:
dacl whitelist spamd < 1
dacl blacklist spamd > 8 msg "Message blocked for spam content."
dacl greylist spamd > 5
That last line shows what the milter can do, though I wouldn't combine
the two like that. The development version can also tarpit (which I
would combine with SA) ... see this commercial tarpit's explanation:
http://mailchannels.com/literature/osterman-research-traffic-shaping.pdf
Big benefit here: that's an SMTP-time REJECT command (which
spamass-milter also does, but amavis, mailscanner, and procmail+spamc
do not). It generates a bounce from the sending server (read: *not*
backscatter), so senders will always know if their message has not
been delivered (and smarter mailing lists, including some that
spammers use, will auto-unsubscribe a bouncing email account).
I flag at 5 and reject at 8. Users can then filter/delete flagged
mail as they see fit without being overwhelmed by volume. I consider
this feature mandatory.
Re: on greylisting...
Posted by Matt <lm...@gmail.com>.
>> For what it's worth, I reconfigured my greylisting relay from a
>> blanket delay to delaying only spamcop neighbors, anything that hits a
>> DNSBL, and any Windows *desktop* (using p0f).
>>
>> The move reduced the fatal delay of 80-90% of my incoming mail down to
>> 64%, which is pretty reasonable given the fact that the inconvenience
>
>
> Some implementations, postgrey for example, keep a whitelist of servers
> known to retry, so once a small number of mails (3 by default iirc) have
> been successfully delivered from a given server (or servers in the same /24
> subnet), it is auto-whitelisted on the basis that there is little point
> continually greylisting servers that you *know* will retry anyway.
>
> This approach works extremely well and after a few weeks normal usage very
> few legitimate mails are delayed by greylisting.
I wrote a perl script that whitelists any servers from greylisting for
6 months that send a message that scores less then 1 by spamassassin.
If it later sends a message that scores greater then 5 it is removed
from the whitelist. Works great. After having a few months to learn
almost no legitimate servers are greylisted.
Matt
Re: on greylisting...
Posted by Ned Slider <ne...@unixmail.co.uk>.
Adam Katz wrote:
>
> For what it's worth, I reconfigured my greylisting relay from a
> blanket delay to delaying only spamcop neighbors, anything that hits a
> DNSBL, and any Windows *desktop* (using p0f).
>
> The move reduced the fatal delay of 80-90% of my incoming mail down to
> 64%, which is pretty reasonable given the fact that the inconvenience
Some implementations, postgrey for example, keep a whitelist of servers
known to retry, so once a small number of mails (3 by default iirc) have
been successfully delivered from a given server (or servers in the same
/24 subnet), it is auto-whitelisted on the basis that there is little
point continually greylisting servers that you *know* will retry anyway.
This approach works extremely well and after a few weeks normal usage
very few legitimate mails are delayed by greylisting.
on greylisting...
Posted by Adam Katz <an...@khopis.com>.
> On 2010-03-30 01:29, Brent Kennedy wrote:
>> Graylisting does work.
Jonas Eckerman wrote:
> I know it works. That's why I said I like it because it stops spam.
> Been using my own implementation for years.
For what it's worth, I reconfigured my greylisting relay from a
blanket delay to delaying only spamcop neighbors, anything that hits a
DNSBL, and any Windows *desktop* (using p0f).
The move reduced the fatal delay of 80-90% of my incoming mail down to
64%, which is pretty reasonable given the fact that the inconvenience
caused by greylisting has all-but vanished: only 3.3% of those delayed
windows desktops makes it through, and more than half of them get
rejected by spamassassin. (I don't have comparable stats from before
the move. Also of note: 90% is a BIG number, so there may be a flaw
in my counting, but since this is relative anyway, it doesn't matter.)
My configuration notes were posted to the milter-greylist wiki at
http://milter-greylist.wikidot.com/using-p0f and my original post is
at http://tech.groups.yahoo.com/group/milter-greylist/message/5496
If I recall correctly, Jonas's implementation also uses p0f and could
therefore benefit from my analysis. The gist of it is that matching
p0f's results with the (perl-compatible) regular expression
/Windows (?:XP|2000(?!SP4)|Vista)/
will safely block only desktops. (Though half of the Windows systems
I see mail from use "Windows 2000 SP4, XP SP1+" and it has to be
excluded from the "desktops" list because there are sooo many MS
Exchange servers out there still running on win2k. I'd love to see
p0f overcome that limitation...)
-Adam
Re: ATTN DEVELOPERS: Mega-Spam
Posted by John Hardin <jh...@impsec.org>.
On Tue, 30 Mar 2010, Jonas Eckerman wrote:
> On 2010-03-30 01:29, Brent Kennedy wrote:
>
>> I think after I turned it on, the botnet plug-in got bored. My stats
>> for it dropped significantly. So that’s my proof it does adversely
>> affect botnets.
>
> No, that's your proof that it has a positive impact on your incoming
> mail stream. It does not prove that it have a significant negative
> impact on the botnets.
I think that TCP tarpits would have more effect on the botnets than
greylisting. However, "more" might still be "not very much", particularly
in the absence of _widespread_ TCP tarpitting of bots.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Homeland Security: Specializing in Tactical Band-aids for Strategic
Problems. -- Eric K. in Bruce Schneier's blog
-----------------------------------------------------------------------
2 days until April Fools' day
Re: ATTN DEVELOPERS: Mega-Spam
Posted by Jonas Eckerman <jo...@frukt.org>.
On 2010-03-30 01:29, Brent Kennedy wrote:
> Graylisting does work.
I know it works. That's why I said I like it because it stops spam. Been
using my own implementation for years.
> I think after I turned it on, the botnet plug-in got bored. My stats for it
> dropped significantly. So that’s my proof it does adversely affect botnets.
No, that's your proof that it has a positive impact on your incoming
mail stream. It does not prove that it have a significant negative
impact on the botnets.
From what I see, botnets seem to have resources to spare.
A lot of sending bots still hasn't adapted to greylisting.
Bots still tries to send to addresses we have been rejecting for 10 years.
I suspect that If the botnets were short on bandwidth and computer
power, the programmers would have fix those issues a long time ago. And
the simple fact that they still haven't adapted to greylisting indicates
that it's impact is not (so far) big enough to care about.
Regards
/Jonas
--
Jonas Eckerman
Fruktträdet & Förbundet Sveriges Dövblinda
http://www.fsdb.org/
http://www.frukt.org/
http://whatever.frukt.org/
RE: ATTN DEVELOPERS: Mega-Spam
Posted by Brent Kennedy <br...@cfl.rr.com>.
Graylisting does work. We have been using SQLGrey
(http://sqlgrey.sourceforge.net/) for three years now. The minute I turned
it on, spam to my junk email folder(what SA used to catch) dropped by 90%.
SQLGrey sits at the MTA level, so it hits the sender when they connect and
before they actually submit email.
Obviously, it does allow them through if they come back, but most botnet
senders do not retry messages or never have the chance.
I think after I turned it on, the botnet plug-in got bored. My stats for it
dropped significantly. So thats my proof it does adversely affect botnets.
I wish I still had the stats graphs for when I turned it on. However, you
can see its affect on my graph here: http://brain.chcfl.com/postfix/ ( noted
as rejections ). I also have active directory setup with the MTA, so no
messages ever hit the server that do not belong nor NDRs generated. If they
try a dictionary attack, they will be on tarpit duty for a long time.
To see the relief on someone's face after they realize they only 10 junk
emails to glance at rather than 100, you see the value of graylisting. I
have put my setup in a few other locations and they also report back to me
that their users are now getting work done rather than parsing emails.
Ya know, this got me thinking. Wonder if I could create a VM with all the
settings and a script to customize the setup. Then organizations could just
deploy the VM. Sort of an all in one deployment. Just update the VM
template every now and then. Ahh but the learning db might be an issue....
oh well just a thought.
-Brent
-----Original Message-----
From: Jonas Eckerman [mailto:jonas_lists@frukt.org]
Sent: Monday, March 29, 2010 6:41 PM
To: John Hardin
Subject: Re: ATTN DEVELOPERS: Mega-Spam
On 2010-03-30 00:12, John Hardin wrote:
> While greylisting will help, it won't spank the offender in that manner.
> It will postpone the message very early in the SMTP exchange, not after
> the body has been received.
Unless the greylisting is done *after* receiving the body. Of course,
this will spank innocent senders as well.
(My selective greylisting implementation for MIMEDefang does this,
originally because some stupid MTAs didn't handle tempfails correctly at
earlier stages... The "selective" stuff keeping delays and spanking of
innocents down.)
BTW: While I like greylisting because it stops a lot of spam, I've never
seen any data substantiating claims that it has a measurable negative
impact on botnets. So I'm not convinced it really does a lot of spanking
of offenders...
Regards
/Jonas
--
Jonas Eckerman
Fruktträdet & Förbundet Sveriges Dövblinda
http://www.fsdb.org/
http://www.frukt.org/
http://whatever.frukt.org/
Re: ATTN DEVELOPERS: Mega-Spam
Posted by Jonas Eckerman <jo...@frukt.org>.
On 2010-03-30 13:31, Kai Schaetzl wrote:
> Jonas Eckerman wrote on Tue, 30 Mar 2010 00:41:01 +0200:
>> Unless the greylisting is done *after* receiving the body. Of course,
>> this will spank innocent senders as well.
> Ooops? It spanks *yourself*.
Not really. It does force us to accept the mail before rejecting it, but
it still rejects a lot of stuff that would otherwise have been scanned
by ClamAV and SpamAssassin before being rejected.
So, while it does not save as much bandwidth and work as greylisting
after RCPT would, it still saves compared to no greylisting. And the
filter does some more stuff. For example:
We also greylist with *one* temporary failure at connect for each host
the first the gateway sees it. This stops more that I irst expecteded
when I tried it.
Once a mail from an MTA has passed the greylist test, that IP is excempt
from the greylist.
We keep tracks of behaviour we don't like. Uknown RCPTs, spam, too many
retries before the greylist period (3 minutes) has passed, etc, etc, and
tempfails hosts at connect based in thsoe counters.
We also make exceptions from the greylist based on DNS whitelists, RDNS
etc so that most mail from real outgoing MTAs pass right through it.
> Good strategy.
My filter works for us.
Most spam is stopped without the gateway having to scan it with
SpamAssassin.
Most ham is passed through without beeing subjected to the greylist or
beeing scanned by SpamAssassin.
And if there still are any stupid MTAs that can't handle tempfails
correctly at earlier stages trying to send mail to us, we have a good
chance of receiving it.
When I first implemented greylisting I did the tempfailing after RCPT,
but some stupid Novell MTA and a security appliance (I think it was
from Syamantec) saw no difference between tamporary failures and
permanen rejects of RCPT TO. And of course one of them they discarded
the response it got from our server when bouncing the mai back to the
sender. Even worse, some other idiotic piece of crap (I forgot what)
reacted to temporary failures at RCPT by simply deleting the mail from
it's queue without notifying anyone.
So, we lost some incoming mail from organizations that for different
reasons didn't just throw out or fix their junk, and I moved the
greylist to after receiving the message data.
Hopefully I could now move it to RCPT, but I actually like beeing able
to log message-id and subject from greylisted mail and I know it works
the way it is now.
Rgards
/Jonas
--
Jonas Eckerman
Fruktträdet & Förbundet Sveriges Dövblinda
http://www.fsdb.org/
http://www.frukt.org/
http://whatever.frukt.org/
Re: ATTN DEVELOPERS: Mega-Spam
Posted by Kai Schaetzl <ma...@conactive.com>.
Jonas Eckerman wrote on Tue, 30 Mar 2010 00:41:01 +0200:
> Unless the greylisting is done *after* receiving the body. Of course,
> this will spank innocent senders as well.
Ooops? It spanks *yourself*. Good strategy.
Kai
--
Get your web at Conactive Internet Services: http://www.conactive.com
Re: ATTN DEVELOPERS: Mega-Spam
Posted by Jonas Eckerman <jo...@frukt.org>.
On 2010-03-30 00:12, John Hardin wrote:
> While greylisting will help, it won't spank the offender in that manner.
> It will postpone the message very early in the SMTP exchange, not after
> the body has been received.
Unless the greylisting is done *after* receiving the body. Of course,
this will spank innocent senders as well.
(My selective greylisting implementation for MIMEDefang does this,
originally because some stupid MTAs didn't handle tempfails correctly at
earlier stages... The "selective" stuff keeping delays and spanking of
innocents down.)
BTW: While I like greylisting because it stops a lot of spam, I've never
seen any data substantiating claims that it has a measurable negative
impact on botnets. So I'm not convinced it really does a lot of spanking
of offenders...
Regards
/Jonas
--
Jonas Eckerman
Fruktträdet & Förbundet Sveriges Dövblinda
http://www.fsdb.org/
http://www.frukt.org/
http://whatever.frukt.org/
RE: ATTN DEVELOPERS: Mega-Spam
Posted by John Hardin <jh...@impsec.org>.
On Mon, 29 Mar 2010, Brent Kennedy wrote:
> My suggestion would be to use graylisting, force them to send that 1MB
> message twice.
While greylisting will help, it won't spank the offender in that manner.
It will postpone the message very early in the SMTP exchange, not after
the body has been received.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Windows Vista: Windows ME for the XP generation.
-----------------------------------------------------------------------
3 days until April Fools' day
RE: ATTN DEVELOPERS: Mega-Spam
Posted by Brent Kennedy <br...@cfl.rr.com>.
Wow, I knew this was coming at some point. I just figured it was too
expensive.
My suggestion would be to use graylisting, force them to send that 1MB
message twice. Course zombie bots don't do that generally, so you would
never even have to deal with it. You could also use the botnet plug-in.
It would be good if SA could handle this though. The above are only
temporary solutions to a bigger problem.
-Brent
-----Original Message-----
From: Charles Gregory [mailto:cgregory@hwcn.org]
Sent: Monday, March 29, 2010 1:09 PM
To: users@spamassassin.apache.org
Subject: ATTN DEVELOPERS: Mega-Spam
Literally, Mega-Spam. I just got a spam with 1MB of images.
My suggestion has been made before, but I would like to ask that it now
be taken a bit more seriously. SA needs an option to allow efficient
'partial' scanning of large e-mails, so that, for example, we can
peform all the valuable header checks, and maybe even scan for URIBL hits
within the first few hundred K of the body?
Is it possible (and easy!) to set a flag that tells SA to stop testing
aganist the body when it reaches a certain byte count.... Or perhaps, if
I understand the docs correctly, most rules only trigger on textual
message parts anyway, so by simply disabling 'full' rules and possbily
'rawbody', we could get the desired result without too much of a
processing hit?
- C
Re: ATTN DEVELOPERS: Mega-Spam
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
Aw, is that shouting really necessary? Oh, yes, it is indeed -- you are
trying to get heard over on the dev list, so you need to be quite loud
from here... ;)
The dev list is what you want.
On Mon, 2010-03-29 at 13:09 -0400, Charles Gregory wrote:
> Literally, Mega-Spam. I just got a spam with 1MB of images.
The largest one I've seen included about 4.5 MByte worth of 7 jpeg
images, the largest one of which 1.2 MByte. And that doesn't even
include the considerable base64 overhead for the mail...
On the other hand: Guess what, I get about one spam per year exceeding
the default size threshold of 500 kByte.
guenther
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}