You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by "Dan Mahoney, System Admin" <da...@prime.gushi.org> on 2007/10/07 11:55:48 UTC
Re: [sa-list] Re: Why did this not hit more? (SPF, DKIM, Ironport,
X-originating-ip)
On Sat, 6 Oct 2007, Rob McEwen wrote:
> Dan,
>
> FWIW... that IP, 220.226.197.15, is currently listed on four spam blacklists
> ("RBLs"):
>
> 1) uceprotect
> 2) no-more-funn
> 3) psbl
> 4) ivmSIP.com (mine)
My problem is: blocklists come and go, and some blocklists, when they
"go", do things like "hang up because they're being flooded, thus slowing
my mail processes" or "flag all mail as spam" or "hand out stale data that
hasn't changed at all in months/years".
If you put out a popular enough blocklist, you're likely to be blocked,
period.
Personally, I'd like it if SA came with a blocklist-feeder tool, where
upon, say, two auto-learns, a blocklist (or SQL database) could be fed.
The docs here:
http://wiki.apache.org/spamassassin/DnsBlocklists?highlight=%28dnsbl%29
Are outdated.
-Dan
>
> The first two are "FP-risky" for outright blocking, but can be useful in a
> scoring environment. The latter two are much more safe for outright
> blocking... particularly ivmSIP.com, which a FP rate that is almost low as
> the FP rate of SpamHaus's lists.
>
> Rob McEwen
>
>
>
>
> Dan Mahoney, System Admin wrote:
>> Message at bottom.
>>
>> I checked on this email. My system is right: it is an spf soft-fail. At
>> this point, ninety nine percent of people who set up SPF are going to be
>> setting ~all and not understanding the difference between ~all and -all.
>> And this did constitute a fail (i.e. a forgery), but there's no rule that
>> hit.
>>
>> We've had the debate before, that SPF alone should not stop spam, but here
>> it is: a legitimate domain hijack and SA isn't hitting?
>>
>> Also, what's up with RDNS_NONE? My sendmail won't accept a connection
>> unless your RDNS resolves, or you send in the domain literal format. I did
>> a quick search and found a few bugs on this.
>>
>> We've already been over DKIM_POLICY_SIGNSOME -- I'm still in favor of
>> making a new rule for the implicit policy (DKIM_NOPOLICY or
>> DKIM_POLICY_ASSUMED_SIGNSSONE) rather than the explicit one.
>>
>> Can we also assume the following...
>>
>> The Ironport-Anti-Spam score is bogus but we have no way of checking the
>> result?
>>
>> The Ironport-AV score is probably also bogus? Are "valid" values for i and
>> a documented somewhere?
>>
>> The X-Originating-IP of 127.0.0.1 is probably accurate (after all, the
>> sending host must have had a 127.1), but useless and either the result of a
>> bug (i.e. a misconfigured mailserver, from which we should not accept), or
>> an intentional attempt to fool filters to believe it's "trusted" (for those
>> systems that check this header) and should be ignored or a rule created?
>>
>>> From patrick12@indiatimes.com Sat Oct 6 05:40:56 2007
>> Return-Path: <pa...@indiatimes.com>
>> X-Spam-Checker-Version: SpamAssassin 3.2.2 (2007-07-23) on quark.gushi.org
>> X-Spam-Level: *
>> X-Spam-Status: No, score=1.4 required=5.0
>> tests=BAYES_50,DKIM_POLICY_SIGNSOME,
>> MISSING_HEADERS,RDNS_NONE autolearn=no version=3.2.2
>> Received: from rx4.indiatimes.com ([220.226.197.15])
>> by prime.gushi.org (8.13.8/8.13.8) with ESMTP id l969eqTG063292
>> for <in...@gushi.org>; Sat, 6 Oct 2007 05:40:54 -0400 (EDT)
>> (envelope-from patrick12@indiatimes.com)
>> Authentication-Results: prime.gushi.org from=patrick12@indiatimes.com;
>> sender-id=softfail; spf=softfail
>> Received: from unknown (HELO tilmb7.indiatimes.com) ([192.168.61.27])
>> by x1.indiatimes.com with ESMTP; 06 Oct 2007 15:07:38 +0530
>> X-IronPort-Anti-Spam-Filtered: true
>> X-IronPort-Anti-Spam-Result: AnoUAJL0BkfAqD0b/2dsb2JhbAAMiRw
>> X-IronPort-AV: i="unknown"; a="17144176:sNHT0"
>> Date: Sat, 6 Oct 2007 14:57:11 +0530 (IST)
>> From: "Mr.Craig McAfee" <pa...@indiatimes.com>
>> Reply-To: "Mr.Craig McAfee" <pa...@yahoo.com.hk>
>> Message-ID:
>> <14...@mbr7.indiatimes.com>
>> Subject: Attn:YOU HAVE WON A PRIZE (1,700,000.00 Euros)!
>> MIME-Version: 1.0
>> X-Originating-IP: [127.0.0.1]
>> Content-Type: text/plain; charset="utf-8"
>> X-Greylist: Default is to whitelist mail, not delayed by
>> milter-greylist-3.0 (prime.gushi.org [0.0.0.0]); Sat, 06 Oct 2007 05:40:56
>> -0400 (EDT)
>> Content-Transfer-Encoding: 8bit
>> X-MIME-Autoconverted: from base64 to 8bit by prime.gushi.org id
>> l969eqTG063292
>> X-Envelope-To: info@gushi.org
>>
>> [ The following text is in the "utf-8" character set. ]
>> [ Your display is set for the "US-ASCII" character set. ]
>> [ Some characters may be displayed incorrectly. ]
>>
>> Attention!!!
>> Your email address has emerged as one of the winner in Euromillions
>> FreeDraws.Prize attached is 1,700,000.00 Euros.Contact Mr Mr Denis Ernest
>> Fing.Email:prize_processing_agent03@yahoo.co.uk
>> with the following information:1, Full Names: 2. Address:3. Age:4. Sex:5.
>> Phone /Fax number: and 6. Country:
>>
>> --
>> My life has changed. What about yours?
>> Log on to the new Indiatimes Mail and Live out of the Inbox!
>>
>> --
>>
>> "Is Gushi a person or an entity?"
>> "Yes"
>>
>> -Bad Karma, August 25th 2001, Ezzi Computers, Quoting himself earler,
>> referring to Gushi
>>
>> --------Dan Mahoney--------
>> Techie, Sysadmin, WebGeek
>> Gushi on efnet/undernet IRC
>> ICQ: 13735144 AIM: LarpGM
>> Site: http://www.gushi.org
>> ---------------------------
>>
>>
>
--
"Of course she's gonna be upset! You're dealing with a woman here Dan,
what the hell's wrong with you?"
-S. Kennedy, 11/11/01
--------Dan Mahoney--------
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---------------------------
Re: [sa-list] RE: Auto-RBL was: Why did this not hit more? (SPF,
DKIM, Ironport, X-originating-ip)
Posted by "Dan Mahoney, System Admin" <da...@prime.gushi.org>.
On Wed, 10 Oct 2007, Bret Miller wrote:
>>> sa-update does NOT feed a local blocklist generated by *my*
>> particular
>>> corpus of spam emails. Think of it as the RBL equivalent of
>>> sitewide-bayes. Or think of it as a way of SA saying "when
>> I get twelve
>>> spams of score 10+ from ip 208.23.118.172...I will feed the
>>> auto-expiring RBL, which *SENDMAIL* works off of, thus keeping my
>>> *SPAMASSASSIN* load lower.
>>
>> How do you call SpamAssassin?
>>
>> If whatever calls SpamAssassin in your setup knows what IP the
>> connecting relay has, it can hopefully also do what you describe
>> above. SpamAssassin doesn't really need to support this (through
>> plugins or anything else) for it to be possible (and feasible).
>
> And I did something very similar as well. The problem I found is that you
> need a very large white list to avoid blocking big ISPs for a sudden flood
> of spam. I ended up rejecting legitimate email far too often from the
> temporary block. I still like the idea and would do it in a second if I
> could change the 5xx reject to a 4xx try later type of block. But I can't'
> without switching to a different MTA.
milter-greylist lets me do this (reject 4XX based on a DNSBL). I've found
it to be highly customizable, if not a bit of a memory pig.
On the other hand, if there is a "big ISP" who is sending me spam...should
they not be blocked, anyway?
-Dan
--
"Long live little fat girls!"
-Recent Taco Bell Ad Slogan, Literally Translated. (Viva Gorditas)
--------Dan Mahoney--------
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---------------------------
RE: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport, X-originating-ip)
Posted by Bret Miller <br...@wcg.org>.
> > sa-update does NOT feed a local blocklist generated by *my*
> particular
> > corpus of spam emails. Think of it as the RBL equivalent of
> > sitewide-bayes. Or think of it as a way of SA saying "when
> I get twelve
> > spams of score 10+ from ip 208.23.118.172...I will feed the
> > auto-expiring RBL, which *SENDMAIL* works off of, thus keeping my
> > *SPAMASSASSIN* load lower.
>
> How do you call SpamAssassin?
>
> If whatever calls SpamAssassin in your setup knows what IP the
> connecting relay has, it can hopefully also do what you describe
> above. SpamAssassin doesn't really need to support this (through
> plugins or anything else) for it to be possible (and feasible).
And I did something very similar as well. The problem I found is that you
need a very large white list to avoid blocking big ISPs for a sudden flood
of spam. I ended up rejecting legitimate email far too often from the
temporary block. I still like the idea and would do it in a second if I
could change the 5xx reject to a 4xx try later type of block. But I can't'
without switching to a different MTA.
Bret
Re: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport,
X-originating-ip)
Posted by Jonas Eckerman <jo...@frukt.org>.
Dan Mahoney, System Admin wrote:
> sa-update does NOT feed a local blocklist generated by *my* particular
> corpus of spam emails. Think of it as the RBL equivalent of
> sitewide-bayes. Or think of it as a way of SA saying "when I get twelve
> spams of score 10+ from ip 208.23.118.172...I will feed the
> auto-expiring RBL, which *SENDMAIL* works off of, thus keeping my
> *SPAMASSASSIN* load lower.
How do you call SpamAssassin?
If whatever calls SpamAssassin in your setup knows what IP the
connecting relay has, it can hopefully also do what you describe
above. SpamAssassin doesn't really need to support this (through
plugins or anything else) for it to be possible (and feasible).
We do something similar (combined with a bunch of other data)
using MIMEDEfang, a *very* customizable sendmail milter.
Our filter checks the IP of each incoming connection against a
database, and rejects the connection if the relay has sent us a
too much spam in proportion to ham, or if it has more than a
certain amount of entries in a specific table (to wich entries
are added for unknown users, spam, and more).
You can try to find more info about our setup by reading the code
at: <http://whatever.frukt.org/mimedefangfilter.text.shtml>
Our filter works with a SQL database, but it should be perfectly
possible to use similar methods to feed a local DNSBL (or write a
DNS server that uses the database as a data source) in order to
be able to use allready existing code for the lookups.
Regards
/Jonas
--
Jonas Eckerman, FSDB & Fruktträdet
http://whatever.frukt.org/
http://www.fsdb.org/
http://www.frukt.org/
Re: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport, X-originating-ip)
Posted by jdow <jd...@earthlink.net>.
From: "Dan Mahoney, System Admin" <da...@prime.gushi.org>
Sent: Monday, 2007, October 08 10:33
> On Mon, 8 Oct 2007, Matus UHLAR - fantomas wrote:
>
>>> On Sat, 6 Oct 2007, Rob McEwen wrote:
>>>> FWIW... that IP, 220.226.197.15, is currently listed on four spam
>>>> blacklists ("RBLs"):
>>>>
>>>> 1) uceprotect
>>>> 2) no-more-funn
>>>> 3) psbl
>>>> 4) ivmSIP.com (mine)
>>
>> On 07.10.07 05:55, Dan Mahoney, System Admin wrote:
>>> My problem is: blocklists come and go, and some blocklists, when they
>>> "go", do things like "hang up because they're being flooded, thus
>>> slowing
>>> my mail processes" or "flag all mail as spam" or "hand out stale data
>>> that
>>> hasn't changed at all in months/years".
>>
>> That's what sa-update is for.
>>
>>> Personally, I'd like it if SA came with a blocklist-feeder tool, where
>>> upon, say, two auto-learns, a blocklist (or SQL database) could be fed.
>>
>> Why do you think people would use them, when they don't already use
>> sa-update which does the same?
>
> sa-update does NOT feed a local blocklist generated by *my* particular
> corpus of spam emails. Think of it as the RBL equivalent of
> sitewide-bayes. Or think of it as a way of SA saying "when I get twelve
> spams of score 10+ from ip 208.23.118.172...I will feed the auto-expiring
> RBL, which *SENDMAIL* works off of, thus keeping my *SPAMASSASSIN* load
> lower. Thus a spam deluge via a dictionary attack that may take hours is
> mitigated in the course of X number of mails.
>
> Which is what I was (off-topicly) asking for,
With procmail you can run any program you want based on features of
messages received. (Now, I'd not run say "startx" from procmail. But I
do use it to detect email from certain people and play a specific tune
for them using "play".) You could develop a little script for performing
that job.
{^_^}
RE: [sa-list] Re: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport,
Posted by Anthony Kamau <an...@anroet.com>.
> -----Original Message-----
> From: Dan Mahoney, System Admin [mailto:danm@prime.gushi.org]
> Sent: Tuesday, 9 October 2007 7:14 AM
> To: Rob McEwen
> Cc: users@spamassassin.apache.org
> Subject: Re: [sa-list] Re: Auto-RBL was: Why did this not hit more? (SPF,
> DKIM, Ironport,
>
> On Mon, 8 Oct 2007, Rob McEwen wrote:
>
> > Therefore, I recommend that you re-think your choices here! Don't let
> your
> > quest for "guaranteed long-term perfection" keep you from making
> > **substantial** progress today!
>
> Rob,
>
> Then help rally the SA team to include those RBLs that you mentioned in
> the stock config.
>
> Also, rally them to update the documentation on the wiki on how to
> configure SA for third-party DNSBL's, because it
> blows (and refers to years-old versions of SA). Yes, I know the point of
> a wiki is that ANYONE can update it, but I'm not about to update it with
> information I don't understand for certain.
You should update the Wiki nevertheless and append a disclaimer of sorts!
Choosing not to update in fear of appearing clueless is just lame! If you
believe that what you are posting is halfway valid, then someone else can
update. This is the sole function of a Wiki as otherwise there'd be no need
for an UPDATE function!!!
.
.
.
Re: [sa-list] Re: Auto-RBL was: Why did this not hit more? (SPF,
DKIM, Ironport,
Posted by Bill Landry <bi...@inetmsg.com>.
Dan Mahoney, System Admin wrote:
> On Mon, 8 Oct 2007, Rob McEwen wrote:
>
>> Therefore, I recommend that you re-think your choices here! Don't let
>> your quest for "guaranteed long-term perfection" keep you from making
>> **substantial** progress today!
>
> Rob,
>
> Then help rally the SA team to include those RBLs that you mentioned in
> the stock config.
>
> Also, rally them to update the documentation on the wiki on how to
> configure SA for third-party DNSBL's, because it blows (and refers to
> years-old versions of SA). Yes, I know the point of a wiki is that
> ANYONE can update it, but I'm not about to update it with information I
> don't understand for certain.
>
> ((Q: This documentation doesn't seem to cover how to configure
> dns-blocklists. It says "Support for these is built-in" but I can't
> believe that all free BL's is called each time a mail is beeing checked.
> There must be a way to configure which to use.
>
> A: You're right. You might look at the [WWW] Mail::SpamAssassin::Conf
> documentation page which I admit doesn't really say how to configure
> which DNSBL to use, or the rules file [WWW] 20_dnsbl_tests.cf, for
> internal details, but no clear examples of how to configure the
> inclusion of various DNSBLs either. For the latest list of DNSBLs you
> want to be using SpamAssassin version 2.63 or 3.0.0-pre2, for the same
> reason that you wouldn't use an out-of-date virus scanner, but that also
> doesn't really have anything to do with the question.))
>
> Finally, rally them to pay attention to the topic I'm proposing here,
> which is: allow users to run their own RBL + feeder so that they can
> auto-rbl and floodgate themselves (and yes, it allows me to combine your
> corpus, plus my corpus, plus HIS corpus) in a scoring config, which is
> FUN...or it lets you say, quite simply "SA said you sent too much spam,
> now sendmail won't listen for X hours per spam run".
>
> <soapbox>
>
> While I've had a long history of getting decent responses from the
> developers on this list some of the time -- nobody has managed to answer
> the questions I've asked in the previous thread:
>
> * can we do something with the ironport headers
>
> * can we do something with the SPF softfail which my MTA registered but
> SA didn't (and why didn't it?)
>
> * can we do something with the X-Originating-IP: 127:1 (is it a legit
> header, or is it there to evade filters?)
>
> * can we fix something about the DKIM_POLICY_SIGNSOME,
>
> * and after I changed the topic: Can we get a plugin that lets us feed
> our own blocklists, currently I get dictionary floods that are enough to
> overload SA (even right now).
Why would you be accepting messages to non-existent users? If you reject these
at the MTA, then SA would never see them and your MTA would not have to deal
with bounces to forged sender addresses (backscatter).
Bill
> and many is the time I've just sent an email out to this list on a given
> topic, seen a lack of useful answer, and shrugged it off.
>
> </soapbox>
>
> --
>
> "Check it out, it's just like Christmas. Except it sucks."
>
> -Jason Seguerra, 3/2/05
>
> --------Dan Mahoney--------
> Techie, Sysadmin, WebGeek
> Gushi on efnet/undernet IRC
> ICQ: 13735144 AIM: LarpGM
> Site: http://www.gushi.org
> ---------------------------
>
Re: [sa-list] Re: Auto-RBL was: Why did this not hit more? (SPF,
DKIM, Ironport,
Posted by Rob McEwen <ro...@invaluement.com>.
Dan,
>Then help rally the SA team to include those RBLs
>that you mentioned in the stock config.
My RBL (ivmSIP.com) wouldn't work as a default value in SA because it is
only available via RSYNC or Zone Transfer to subscribers (or...
currently... "testers" who have specifically requested access).
The other weird thing is that I use SA as a "helper app" in my spam
filtering and I've custom written my own spam filter. Mostly, I still
include SA in the mix for SARE rules (& other rules), as well as
checksum filtering like RAZAR, etc. But I've turned off all RBL & URIBL
filtering in SA because I do those on my own and, most of the time, SA
isn't even needed.
As a result, I pay very little attention to many of the implementation
details of "RBLs" in SA since I don't personally use them in SA. I have
enough to worry about without these extra details. However, I'll be
happy to share some tips that might help others or the SA folks with
possible improvements in future versions.
First, one thing that I did years ago (and continue to do) is that I'm
always carefully reviewing lists that I might potentially use and/or am
already using. For example, if I notice that a particular dnsbl is
hitting on more and more messages which ultimately score under the spam
threshold and, upon examination, I verify that most or all of these
really are legit, them I'm at least going to lower the points assigned
to hits on that dnsbl... and I might even remove that dnsbl from my spam
filtering altogether.
If, on the other hand, I find that ALL such messages really were spam, I
might start increasing the points given to that particular list,
assuming that I'm not also seeing some FPs from that list.
Next, if I see a spam (that wasn't sent from a legit ISPs mailserver)
and it scored rather low, I'll then take that IP and run it against a
spam blacklist checker (dnsstuff, robtex, etc) to see if there are any
RBLs that would have caught it, but that I'm not using yet. (Of course,
I ignore various FP-ridden lists like APEWS in that search.) If I see a
pattern whereby a particular list consistently hits on IPs that scored
too low in my spam filtering, I might then add that dnsbl to my
filtering... starting off with a low score... then double-checking for
FPs... then bumping the score up depending on how little FPs there are.
(in this case, I'm calling any "hits" on legit messages a "FP", but, at
this stage, these will generate too low a score to outright block and
this "FP" really did get delivered to the inbox.)
Doing this, over the years, I've added a good mix of RBLs with very fine
tuned scoring (in my own spam filtering program, not referring to SA).
At one point, I noticed that many of the more aggressive dnsbls are
really really good at catching new IPs, but have too many FPs. As a
result, I have to keep their "score" low. But it seemed such a shame
because these IPs were taking too long to get on the FP-safe dnsbls.
Then I noticed that, many times, three or four of the more aggressive
RBLs would quickly hit on the same spammer's IP, where that IP that
wasn't yet on SpamHaus, etc... then... if a few lists hit on that new
spammer' IP, chances were, it was worthy of blocking in comparison to if
just one list hit on it... so much so that the score really needed to be
higher than merely the sum of the FP-risky dnsbl's scores.
As a result, I changed my formula so that I took into account the number
of dnsbls that hit on that IP as well as the score. (it was something
like.. for every added dnsbld "hit" the overall RBL score would get
increased by an additional 10% or 20%)... next, I adjusted down some of
the "raw" scores so as to not allow the RBL scoring to get out of
control. IOW... the whole really was worth more than the sum of its
parts! Get it?
Of course, even then, I have extensive whitelisting of IPs that I have
placed in front of this... both my own (that I've put literally
thousands of hours into!) and third parties. Currently, my own IP
blacklist isn't (yet) on dnsstuff or robtex... but if it something like
it were there and produced by someone else... I would have spotted it in
that systematic checking that I described and I'd have been thrilled at
its results... IOW... I created a product that I myself would have
greatly desired to have if it had been created and distributed by
someone else. I probably would have been one the first subscribers.. had
this been someone else's product. (Why? Because my RBL provides that
same "fast reacting aggressiveness"... just without the FPs!)
Still, besides my own RBL's "subscription" barrier to inclusion... other
lists which also require RSYNC access would not be able to come
"preinstalled" in SA since they too need a little TLC to get up and
running in one's spam filtering environment. These couldn't be used "out
of the box" without some configuring of various programs on one's
server. Something else to ponder.
I hope this is beneficial and helps future SA versions! Doing all of
this, I believe I've taken the "RBL" portion of my spam filtering to a
level that is beyond what many thought possible.
Rob McEwen
Re: [sa-list] Re: Auto-RBL was: Why did this not hit more? (SPF,
DKIM, Ironport,
Posted by "Dan Mahoney, System Admin" <da...@prime.gushi.org>.
On Mon, 8 Oct 2007, Rob McEwen wrote:
> Therefore, I recommend that you re-think your choices here! Don't let your
> quest for "guaranteed long-term perfection" keep you from making
> **substantial** progress today!
Rob,
Then help rally the SA team to include those RBLs that you mentioned in
the stock config.
Also, rally them to update the documentation on the wiki on how to
configure SA for third-party DNSBL's, because it
blows (and refers to years-old versions of SA). Yes, I know the point of
a wiki is that ANYONE can update it, but I'm not about to update it with
information I don't understand for certain.
((Q: This documentation doesn't seem to cover how to configure
dns-blocklists. It says "Support for these is built-in" but I can't
believe that all free BL's is called each time a mail is beeing checked.
There must be a way to configure which to use.
A: You're right. You might look at the [WWW] Mail::SpamAssassin::Conf
documentation page which I admit doesn't really say how to configure which
DNSBL to use, or the rules file [WWW] 20_dnsbl_tests.cf, for internal
details, but no clear examples of how to configure the inclusion of
various DNSBLs either. For the latest list of DNSBLs you want to be using
SpamAssassin version 2.63 or 3.0.0-pre2, for the same reason that you
wouldn't use an out-of-date virus scanner, but that also doesn't really
have anything to do with the question.))
Finally, rally them to pay attention to the topic I'm proposing here,
which is: allow users to run their own RBL + feeder so that they can
auto-rbl and floodgate themselves (and yes, it allows me to combine your
corpus, plus my corpus, plus HIS corpus) in a scoring config, which is
FUN...or it lets you say, quite simply "SA said you sent too much spam,
now sendmail won't listen for X hours per spam run".
<soapbox>
While I've had a long history of getting decent responses from the
developers on this list some of the time -- nobody has managed to answer
the questions I've asked in the previous thread:
* can we do something with the ironport headers
* can we do something with the SPF softfail which my MTA registered but SA
didn't (and why didn't it?)
* can we do something with the X-Originating-IP: 127:1 (is it a legit
header, or is it there to evade filters?)
* can we fix something about the DKIM_POLICY_SIGNSOME,
* and after I changed the topic: Can we get a plugin that lets us feed our
own blocklists, currently I get dictionary floods that are enough to
overload SA (even right now).
and many is the time I've just sent an email out to this list on a given
topic, seen a lack of useful answer, and shrugged it off.
</soapbox>
--
"Check it out, it's just like Christmas. Except it sucks."
-Jason Seguerra, 3/2/05
--------Dan Mahoney--------
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---------------------------
Re: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport,
X-originating-ip)
Posted by Rob McEwen <ro...@invaluement.com>.
On 07.10.07 05:55, Dan Mahoney, System Admin wrote:
>My problem is: blocklists come and go, and some blocklists, when they
>"go", do things like "hang up because they're being flooded, thus slowing
>my mail processes" or "flag all mail as spam" or "hand out stale data
that
>hasn't changed at all in months/years".
Dan,
Of all your worries, "flag all mail as spam" is by far the absolute
worst. But even that is very rare and, when it does happen, typically,
the blacklist had already been discontinued for months or years.
Nevertheless, the ONLY reason I've seen the "flag all mail as spam"
tactic used is to reduce the load on a DNS server or some other kind of
server where the owner of that domain seeks relief from the onslaught of
continued queries long past the life of the dnsbl.
But this "worst case" scenario will never occur with these two
blacklists that I referred to because both of them distribute their data
via RSYNC... so there really isn't any server to protect from direct
queries that happen years/months after the list is discontinued.
Your concern about "hang up because they're being flooded, thus slowing
my mail processes" is also not valid because when using RSYNC, **you**
are the one serving this data to your spam filtering... so you aren't
dependent on the speed or reliability of a 3rd party DNS server
interacting in real time with your spam filter (or lack thereof). In
fact, I pursue RSYNC whenever possible because I can't stand the idea of
my spam filtering being depending on the speed/reliability of
overburdened 3rd party DNS servers! (sure, I use local DNS caching...
and that helps... but some lookups are not going to be in the cache yet.)
Therefore, about the only concern that is left is (a) stale data from an
abandoned dnsbl... which is possible from an RSYNC-accessed list... as
what seems to have happened with tqmcube.com recently??, or (b) the
list's RSYNC server gets overloaded.
But, really, at some point, EVERYTHING in your life involves risk. You
take a risk every time you drive your care that someone heavily drunk is
going to swerve across the double line and hit you head-on. And I don't
think ANY spam filtering tactic is guaranteed to be "set it and forget
it" year after year after year. They all require some amount of
monitoring... if only, at the least, to ensure that some kind of
misconfiguration hasn't occurred on the consumer of the blacklist's end.
So the question is... are the benefits worth the risk? And what are the
probabilities involved?
In the case of "psbl", their FP rate has gotten really low in recent
months and they catch much spam that others miss and they've been around
for many years... demonstrating a pattern of reliability.
ivmSIP.com also catches much spam with an extremely low FP rate... and
ivmSIP is the "new kid on the block"... but I've found that a common
thread of many of the "dead RBLs" is that they were run on a volunteer
basis and the volunteer simply burned out and quit. This won't happen
with ivmSIP.com because ivmSIP will soon become a subscription-based RBL
where there is an economic incentive for the list to stay current and
for the paying subscribers to be satisfied with the service... something
these other "dead RBLs" were lacking! This will also prevent ivmSIP's
RSYNC server from getting overloaded or from having to limit
subscriber's data update frequency, as many others lists are forced to
do to keep their RSYNC servers from getting overloaded from TONS of
"free subscribers".
Therefore, I recommend that you re-think your choices here! Don't let
your quest for "guaranteed long-term perfection" keep you from making
**substantial** progress today!
Rob McEwen
Re: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport,
X-originating-ip)
Posted by bg...@idcomm.com.
Dan Mahoney, System Admin wrote:
> On Tue, 9 Oct 2007, Steven Kurylo wrote:
>
>>> Parsing the SA logs would be easy, but the connecting IP isn't listed
>>> there.
>> As I mentioned, I'm parsing exim's logs. It contains the spam score and the
>> IP address.
>
> Oh, that's true enough. I was musing on parsing my own logfiles as
> opposed to plugins. Not enough info since I'm rejecting at the procmail
> level, not the MTA (sendmail) level.
>
> -Dan
message-id from spam(d/assassin) log line, message-id -> queue-id,
queue-id -> connecting IP.
Shouldn't be too hard to write in perl, just have to keep track of
active (hasn't finished local delivery) IP/QID/MID triples.
Also depending on your MTA you may be able to pass the connecting IP to
procmail.
Re: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport,
X-originating-ip)
Posted by "Dan Mahoney, System Admin" <da...@prime.gushi.org>.
On Tue, 9 Oct 2007, Steven Kurylo wrote:
>> Parsing the SA logs would be easy, but the connecting IP isn't listed
>> there.
> As I mentioned, I'm parsing exim's logs. It contains the spam score and the
> IP address.
Oh, that's true enough. I was musing on parsing my own logfiles as
opposed to plugins. Not enough info since I'm rejecting at the procmail
level, not the MTA (sendmail) level.
-Dan
--
"Ca. Tas. Tro. Phy."
-John Smedley, March 28th 1998, 3AM
--------Dan Mahoney--------
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---------------------------
Re: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport,
X-originating-ip)
Posted by Steven Kurylo <st...@aviawest.com>.
> Parsing the SA logs would be easy, but the connecting IP isn't listed
> there.
As I mentioned, I'm parsing exim's logs. It contains the spam score and
the IP address.
Re: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport,
X-originating-ip)
Posted by "Dan Mahoney, System Admin" <da...@prime.gushi.org>.
On Tue, 9 Oct 2007, Steven Kurylo wrote:
>> Or think of it as a way of SA saying "when I get twelve spams of score 10+
>> from ip 208.23.118.172...I will feed the auto-expiring RBL, which
>> *SENDMAIL* works off of, thus keeping my *SPAMASSASSIN* load lower. Thus a
>> spam deluge via a dictionary attack that may take hours is mitigated in the
>> course of X number of mails.
> I already do something similar, but I haven't bothered to take it quite that
> far yet.
>
> I use fail2ban to parse my exim logs. If an IP address hits more than 5
> invalid accounts in 5 minutes, the IP is banned (fail2ban uses iptables) for
> 24 hours. As well if an IP address, which is listed on spamhause, hits me
> more than twice in 5 minutes it is banned for 24 hours. Granted neither of
> these cases usually end up getting messages as far as spamassassin.
>
> I've managed to drastically reduce the amount of simultaneous connections
> using this method; which was overloading the server. The next step would be
> to add the "when I get twelve spams of score 10+ from [...]" parsing. Though
> I hadn't thought of trying my hand at a SA plugin, I may do that.
Parsing the SA logs would be easy, but the connecting IP isn't listed
there.
-Dan
--
"Man, this is such a trip"
-Dan Mahoney, October 25, 1997
--------Dan Mahoney--------
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---------------------------
Re: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport,
X-originating-ip)
Posted by Steven Kurylo <st...@aviawest.com>.
> Or think of it as a way of SA saying "when I get twelve spams of
> score 10+ from ip 208.23.118.172...I will feed the auto-expiring RBL,
> which *SENDMAIL* works off of, thus keeping my *SPAMASSASSIN* load
> lower. Thus a spam deluge via a dictionary attack that may take hours
> is mitigated in the course of X number of mails.
I already do something similar, but I haven't bothered to take it quite
that far yet.
I use fail2ban to parse my exim logs. If an IP address hits more than 5
invalid accounts in 5 minutes, the IP is banned (fail2ban uses iptables)
for 24 hours. As well if an IP address, which is listed on spamhause,
hits me more than twice in 5 minutes it is banned for 24 hours. Granted
neither of these cases usually end up getting messages as far as
spamassassin.
I've managed to drastically reduce the amount of simultaneous
connections using this method; which was overloading the server. The
next step would be to add the "when I get twelve spams of score 10+ from
[...]" parsing. Though I hadn't thought of trying my hand at a SA
plugin, I may do that.
Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport,
X-originating-ip)
Posted by "Dan Mahoney, System Admin" <da...@prime.gushi.org>.
On Mon, 8 Oct 2007, Matus UHLAR - fantomas wrote:
>> On Sat, 6 Oct 2007, Rob McEwen wrote:
>>> FWIW... that IP, 220.226.197.15, is currently listed on four spam
>>> blacklists ("RBLs"):
>>>
>>> 1) uceprotect
>>> 2) no-more-funn
>>> 3) psbl
>>> 4) ivmSIP.com (mine)
>
> On 07.10.07 05:55, Dan Mahoney, System Admin wrote:
>> My problem is: blocklists come and go, and some blocklists, when they
>> "go", do things like "hang up because they're being flooded, thus slowing
>> my mail processes" or "flag all mail as spam" or "hand out stale data that
>> hasn't changed at all in months/years".
>
> That's what sa-update is for.
>
>> Personally, I'd like it if SA came with a blocklist-feeder tool, where
>> upon, say, two auto-learns, a blocklist (or SQL database) could be fed.
>
> Why do you think people would use them, when they don't already use
> sa-update which does the same?
sa-update does NOT feed a local blocklist generated by *my* particular
corpus of spam emails. Think of it as the RBL equivalent of
sitewide-bayes. Or think of it as a way of SA saying "when I get twelve
spams of score 10+ from ip 208.23.118.172...I will feed the auto-expiring
RBL, which *SENDMAIL* works off of, thus keeping my *SPAMASSASSIN* load
lower. Thus a spam deluge via a dictionary attack that may take hours is
mitigated in the course of X number of mails.
Which is what I was (off-topicly) asking for,
-Dan
--
"I'll commit ritual suicide before I whore myself out to Disney."
--Emi Bryant
April 26, 2004
On the animation industry
--------Dan Mahoney--------
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---------------------------
Re: [sa-list] Re: Why did this not hit more? (SPF, DKIM, Ironport, X-originating-ip)
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
> On Sat, 6 Oct 2007, Rob McEwen wrote:
> >FWIW... that IP, 220.226.197.15, is currently listed on four spam
> >blacklists ("RBLs"):
> >
> >1) uceprotect
> >2) no-more-funn
> >3) psbl
> >4) ivmSIP.com (mine)
On 07.10.07 05:55, Dan Mahoney, System Admin wrote:
> My problem is: blocklists come and go, and some blocklists, when they
> "go", do things like "hang up because they're being flooded, thus slowing
> my mail processes" or "flag all mail as spam" or "hand out stale data that
> hasn't changed at all in months/years".
That's what sa-update is for.
> Personally, I'd like it if SA came with a blocklist-feeder tool, where
> upon, say, two auto-learns, a blocklist (or SQL database) could be fed.
Why do you think people would use them, when they don't already use
sa-update which does the same?
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Fucking windows! Bring Bill Gates! (Southpark the movie)