You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Marc Perkel <ma...@perkel.com> on 2007/08/16 05:11:24 UTC
Question - How many of you run ALL your email through SA?
As opposed to preprocessing before using SA to reduce the load. (ie.
using blacklist and whitelist before SA)
Re: Question - How many of you run ALL your email through SA?
Posted by Marc Perkel <ma...@perkel.com>.
Matus UHLAR - fantomas wrote:
> On 16.08.07 15:44, Mike Jackson wrote:
>
>> RBL: dynablock.njabl.org
>>
>
> this one is obsole and you should not use it. It was imported to the
> SpamHaus PBL and is not maintained by NJABL anymore
>
>
>
Thanks for that. Good to know.
Re: Question - How many of you run ALL your email through SA?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 16.08.07 15:44, Mike Jackson wrote:
> RBL: dynablock.njabl.org
this one is obsole and you should not use it. It was imported to the
SpamHaus PBL and is not maintained by NJABL anymore
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
I intend to live forever - so far so good.
Re: Question - How many of you run ALL your email through SA?
Posted by Mike Jackson <mj...@barking-dog.net>.
> As opposed to preprocessing before using SA to reduce the load. (ie.
> using blacklist and whitelist before SA)
SMTP -> Sendmail, which uses these to reject:
GreetPause
RBL: zen.spamhaus.org
RBL: dynablock.njabl.org
RBL: dsn.rfc-ignorant.org
RBL: bogusmx.rfc-ignorant.org
RBL: bl.spamcop.net
SPF
Sendmail -> Procmail
feed "spamtrap" addresses or bad domains to sa-learn, then throw away
don't filter postmaster
Procmail -> SpamAssassin
send to /dev/null anything autolearned
Re: Question - How many of you run ALL your email through SA?
Posted by Tom Q Citizen <to...@bay-online-media.com>.
Marc Perkel wrote:
> As opposed to preprocessing before using SA to reduce the load. (ie.
> using blacklist and whitelist before SA)
>
I run everything through SA. I really need to upgrade from 3.1.8 to
3.2.3. :)
Peace...
Tom
RE: Question - How many of you run ALL your email through SA?
Posted by Skip Brott <sb...@dmp.com>.
> As opposed to preprocessing before using SA to reduce the load. (ie.
> using blacklist and whitelist before SA)
I do. I have so few issues with SA rulesets (and sare rulesets) with FPs or
missing spam [other than when new variations come in] that I'd rather put
the load on my server. I don't agree with the methodology of sites like
spamhaus & spamcop so I only use the scoring rules built into SA rather than
just simply give blacklisting control to another service.
- Skip
Re: Question - How many of you run ALL your email through SA?
Posted by "C. Bensend" <be...@bennyvision.com>.
> As opposed to preprocessing before using SA to reduce the load. (ie.
> using blacklist and whitelist before SA)
With the exception of rejecting during the SMTP conversation for
viruses/unknown users, yes, everything goes through SA.
Benny
--
"This officer's men seem to follow him merely out of idle curiosity."
-- Sandhurst officer cadet evaluation
Re: Question - How many of you run ALL your email through SA?
Posted by Jari Fredriksson <ja...@iki.fi>.
> As opposed to preprocessing before using SA to reduce the
> load. (ie. using blacklist and whitelist before SA)
I do whitelist certain ham addresses, and do not pass them to SA. Also all mailinglists like this will not get SpamAssassined.
Re: Question - How many of you run ALL your email through SA?
Posted by Tom Q Citizen <to...@bay-online-media.com>.
Steven Stern wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 08/15/2007 10:11 PM, Marc Perkel wrote:
>
>> As opposed to preprocessing before using SA to reduce the load. (ie.
>> using blacklist and whitelist before SA)
>>
>>
>>
> We do, except for virus processing through clamav.
>
>
Actually, we do this as well. :)
Peace...
Tom
Re: Question - How many of you run ALL your email through SA?
Posted by Steven Stern <su...@sterndata.com>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 08/15/2007 10:11 PM, Marc Perkel wrote:
> As opposed to preprocessing before using SA to reduce the load. (ie.
> using blacklist and whitelist before SA)
>
>
We do, except for virus processing through clamav.
- --
Steve
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
iD4DBQFGw8J0eERILVgMyvARAt93AJdoegUYzWnM+uZyzORiec2+9+2rAJ4jDAYR
AT4QOwVlphvPjs0fc4D+FQ==
=8GGT
-----END PGP SIGNATURE-----
Re: Question - How many of you run ALL your email through SA?
Posted by "Eric A. Hall" <eh...@ehsco.com>.
On 8/15/2007 11:11 PM, Marc Perkel wrote:
> As opposed to preprocessing before using SA to reduce the load. (ie.
> using blacklist and whitelist before SA)
All email sent to port 25 goes through SA for processing. Postfix has a
couple of regular expressions and some behavioral stuff (invalid commands,
invalid recipients, etc), but otherwise it just looks for the spam score
and if its too high the transfer is rejected.
--
Eric A. Hall http://www.ehsco.com/
Internet Core Protocols http://www.oreilly.com/catalog/coreprot/
Re: Question - How many of you run ALL your email through SA?
Posted by martin f krafft <ma...@madduck.net>.
also sprach Marc Perkel <ma...@perkel.com> [2007.08.16.0511 +0200]:
> As opposed to preprocessing before using SA to reduce the load. (ie. using
> blacklist and whitelist before SA)
I have a bunch of postfix sanity checks, including RBLs running
first. Then, everything is fed to spamc, which --pipe-to's to
dovecot's deliver.
For my personal setup (I receive about as much email as the 600
other users combined :>), I use a relatively complex procmail setup.
This runs the spamfilter on all messages except
- blacklisted messages
- messages fitting regexps which exempt them from spamfiltering
(e.g. admin stuff, debian bugs, addresses used on certain
websites, incl. ebay, rss2email mails, and test messages)
- duplicates
- messages being reprocessed (train-on-error via IMAP)
The procmail ruleset is rather complex, but it allows me to do cool
stuff, among which:
- defer local delivery, while I am working on the filter.
- blacklist according to regexps.
- kill duplicate list messages sent to my personal address.
- use crm114+SA and combine the two such that SA can train crm114
if the score is high/low enough.
- train-on-error based on spamtraps or moving messages between
IMAP folders.
- filter mail not in-reply-to one of my own messages, or
mentioning my name, or certain keywords.
- archive mail with gmail and a local lurker instance.
If you are interested, I'll happily share it with you. I don't want
to post it here though because it contains my black/whitelists etc.
and is thus not destined for the general public.
--
martin; (greetings from the heart of the sun.)
\____ echo mailto: !#^."<*>"|tr "<*> mailto:" net@madduck
"here i was all convinced that if i sleep all day, bug counts go
down, and if I work all day, they go up, so much for that theory."
-- lars wirzenius
spamtraps: madduck.bogus@madduck.net
Re: Question - How many of you run ALL your email through SA?
Posted by Aaron Wolfe <aa...@gmail.com>.
On 8/16/07, Dave Mifsud <da...@um.edu.mt> wrote:
> On 16/08/07 08:45, Aaron Wolfe wrote:
> > I agree and have yet another similar setup here. We reject about 80%
> > as well, which helps reduce the load on the servers and on the users
> > who manage their quarantines. We allow users to choose whether to use
> > no filtering, the "pre SA", reject filtering only, or full content
> > filtering with SA. A surprising number prefer to use just the more
> > basic checks and deal with what gets through with their mua.
> >
> > -Aaron
> >
>
> What's the default option for users?
>
A good question... it's chosen by each domain's administrator when
they sign up. There is no default, but there probably is a strong
correlation between what the admin chooses and what the users are
using :) I should probably run some statistics on that.
-Aaron
> Dave
> --
> Dave Mifsud
> Systems Engineer
> Computing Services Centre
> University of Malta
>
> CSC Tel: (+356) 2340 3004 CSC Fax: (+356) 21 343 397
>
>
Re: Question - How many of you run ALL your email through SA?
Posted by Dave Mifsud <da...@um.edu.mt>.
On 16/08/07 08:45, Aaron Wolfe wrote:
> I agree and have yet another similar setup here. We reject about 80%
> as well, which helps reduce the load on the servers and on the users
> who manage their quarantines. We allow users to choose whether to use
> no filtering, the "pre SA", reject filtering only, or full content
> filtering with SA. A surprising number prefer to use just the more
> basic checks and deal with what gets through with their mua.
>
> -Aaron
>
What's the default option for users?
Dave
--
Dave Mifsud
Systems Engineer
Computing Services Centre
University of Malta
CSC Tel: (+356) 2340 3004 CSC Fax: (+356) 21 343 397
Re: Question - How many of you run ALL your email through SA?
Posted by Aaron Wolfe <aa...@gmail.com>.
On 8/16/07, Matthias Haegele <mh...@linuxrocks.dyndns.org> wrote:
> John Rudd schrieb:
> > Marc Perkel wrote:
> >> As opposed to preprocessing before using SA to reduce the load. (ie.
> >> using blacklist and whitelist before SA)
> >>
> >>
> >
> >
> > I do not.
> >
> > (greet-pause of 5 seconds; zen and dsbl as blacklists; local access type
> > blocks; dangerous attachment filename blocker; and then clamav with
> > Sanesecurity, MSRBL, MBL signatures; all of those _reject_ messages
> > during the SMTP session before Spam Assassin gets to see them)
>
> Nearly same setup as John. If you have the opportunity to block at MTA
> level i think u *really should do this*. (Its around 80% rejects here).
> Additionaly i block some TLDs like .ar|br|cl|ru|pl|jp|hu which i dont
> have regular mail contact here ...
> btw: MTA is Postfix.
>
I agree and have yet another similar setup here. We reject about 80%
as well, which helps reduce the load on the servers and on the users
who manage their quarantines. We allow users to choose whether to use
no filtering, the "pre SA", reject filtering only, or full content
filtering with SA. A surprising number prefer to use just the more
basic checks and deal with what gets through with their mua.
-Aaron
Re: Question - How many of you run ALL your email through SA?
Posted by Matthias Haegele <mh...@linuxrocks.dyndns.org>.
John Rudd schrieb:
> Marc Perkel wrote:
>> As opposed to preprocessing before using SA to reduce the load. (ie.
>> using blacklist and whitelist before SA)
>>
>>
>
>
> I do not.
>
> (greet-pause of 5 seconds; zen and dsbl as blacklists; local access type
> blocks; dangerous attachment filename blocker; and then clamav with
> Sanesecurity, MSRBL, MBL signatures; all of those _reject_ messages
> during the SMTP session before Spam Assassin gets to see them)
Nearly same setup as John. If you have the opportunity to block at MTA
level i think u *really should do this*. (Its around 80% rejects here).
Additionaly i block some TLDs like .ar|br|cl|ru|pl|jp|hu which i dont
have regular mail contact here ...
btw: MTA is Postfix.
--
Grüsse/Greetings
MH
Dont send mail to: ubecatcher@linuxrocks.dyndns.org
--
Re: Question - How many of you run ALL your email through SA?
Posted by John Rudd <jr...@ucsc.edu>.
Marc Perkel wrote:
>
>
> John Rudd wrote:
>> Marc Perkel wrote:
>>> As opposed to preprocessing before using SA to reduce the load. (ie.
>>> using blacklist and whitelist before SA)
>>>
>>>
>>
>>
>> I do not.
>>
>> (greet-pause of 5 seconds; zen and dsbl as blacklists; local access
>> type blocks; dangerous attachment filename blocker; and then clamav
>> with Sanesecurity, MSRBL, MBL signatures; all of those _reject_
>> messages during the SMTP session before Spam Assassin gets to see them)
>>
>
> Do you check any whitelists to bypass ham around SA?
>
If SMTP-AUTH or webmail is used, then the blacklists, local access
blocks, dangerous attachment blocker, and spam assassin checks are all
skipped.
I _can_ whitelist different individual addresses from any check, but I
don't.
Re: Question - How many of you run ALL your email through SA?
Posted by Marc Perkel <ma...@perkel.com>.
John Rudd wrote:
> Marc Perkel wrote:
>> As opposed to preprocessing before using SA to reduce the load. (ie.
>> using blacklist and whitelist before SA)
>>
>>
>
>
> I do not.
>
> (greet-pause of 5 seconds; zen and dsbl as blacklists; local access
> type blocks; dangerous attachment filename blocker; and then clamav
> with Sanesecurity, MSRBL, MBL signatures; all of those _reject_
> messages during the SMTP session before Spam Assassin gets to see them)
>
Do you check any whitelists to bypass ham around SA?
Re: Question - How many of you run ALL your email through SA?
Posted by John Rudd <jr...@ucsc.edu>.
Marc Perkel wrote:
> As opposed to preprocessing before using SA to reduce the load. (ie.
> using blacklist and whitelist before SA)
>
>
I do not.
(greet-pause of 5 seconds; zen and dsbl as blacklists; local access type
blocks; dangerous attachment filename blocker; and then clamav with
Sanesecurity, MSRBL, MBL signatures; all of those _reject_ messages
during the SMTP session before Spam Assassin gets to see them)
Re: Question - How many of you run ALL your email through SA?
Posted by John Rudd <jr...@ucsc.edu>.
mailinglist@krausam.de wrote:
>> It's interesting to me that your chosen example of "doing it right" is
>> in fact doing 2 very wrong things (bl.spamcop.net as a blacklist, and
>> sender callback).
>
> What's the problem with bl.spamcop.net?
a) poor quality control on the part of spamcop leads to lots of false
positives (listing the wrong host due to improper analysis of the
received header, listing due to inaccurate reports, etc.)
b) poor policies on the part of spamcop leads to lots of false positives
(listing due to various types of autoresponders, etc.)
c) last I checked (which may have changed) even THEY don't recommend
using their blacklist as an actual MTA blacklist, but instead they
recommend it for things like SA blacklist checks (ie. use it for
scoring, not for outright blocking).
Re: Question - How many of you run ALL your email through SA?
Posted by Marc Perkel <ma...@perkel.com>.
mailinglist@krausam.de wrote:
>> It's interesting to me that your chosen example of "doing it right" is
>> in fact doing 2 very wrong things (bl.spamcop.net as a blacklist, and
>> sender callback).
>>
>
> What's the problem with bl.spamcop.net?
>
>
I use spamcop.net but in my opinion it's not quite good enough to block
on. What I do is that I have several levels of MX and I do a temp error
on the lowest MX and accept if they come back on a higher MX.
Re: Question - How many of you run ALL your email through SA?
Posted by ma...@krausam.de.
> It's interesting to me that your chosen example of "doing it right" is
> in fact doing 2 very wrong things (bl.spamcop.net as a blacklist, and
> sender callback).
What's the problem with bl.spamcop.net?
Re: Question - How many of you run ALL your email through SA?
Posted by Marc Perkel <ma...@perkel.com>.
Rob Sterenborg wrote:
> Marc Perkel wrote:
>
>>> Marc Perkel wrote:
>>>
>>>> OK - it's interesting that of all of you who responded this
>>>> is the only person who is doing it right. I have to say that
>>>> I'm somewhat surprised that so few people are preprocessing
>>>> their email to reduce the SA load. As we all know SA is very
>>>> processor and memory expensive.
>>>>
>>>>
>> Aaron Wolfe wrote:
>>
>>> What was the motivation behind your original post? What were
>>> you hoping to learn?
>>>
>>>
>> I was just curious based on some of the comments here that
>> people really were running everything through SA. I'm just
>> surprised that so many were doing that.
>>
>
> How many did you count that do *not* filter email prior to passing it to
> SA, opposed to people that do?
> I think I saw (and that there are) more people that pre-filter than that
> don't (no, I didn't count; that's just what I think that I saw/think).
> In fact, I didn't even see enough replies to base any conclusion upon:
> not everyone that's subscribed to this list replied and even then: the
> world is (much) larger than this list.
>
> Further, as some already said: what may be right for you, could be, but
> doesn't have to be, right for anyone else. I do not think I need you to
> tell me if I'm doing it the right way, which is what you are doing.
>
>
> Grts,
> Rob
>
>
Yes - it does seem now that there are a lot more prefiltering that I had
originally though.
RE: Question - How many of you run ALL your email through SA?
Posted by Rob Sterenborg <R....@netsourcing.nl>.
Marc Perkel wrote:
>> Marc Perkel wrote:
>>> OK - it's interesting that of all of you who responded this
>>> is the only person who is doing it right. I have to say that
>>> I'm somewhat surprised that so few people are preprocessing
>>> their email to reduce the SA load. As we all know SA is very
>>> processor and memory expensive.
>>>
> Aaron Wolfe wrote:
>> What was the motivation behind your original post? What were
>> you hoping to learn?
>>
> I was just curious based on some of the comments here that
> people really were running everything through SA. I'm just
> surprised that so many were doing that.
How many did you count that do *not* filter email prior to passing it to
SA, opposed to people that do?
I think I saw (and that there are) more people that pre-filter than that
don't (no, I didn't count; that's just what I think that I saw/think).
In fact, I didn't even see enough replies to base any conclusion upon:
not everyone that's subscribed to this list replied and even then: the
world is (much) larger than this list.
Further, as some already said: what may be right for you, could be, but
doesn't have to be, right for anyone else. I do not think I need you to
tell me if I'm doing it the right way, which is what you are doing.
Grts,
Rob
Re: Question - How many of you run ALL your email through SA?
Posted by DAve <da...@pixelhammer.com>.
Marc Perkel wrote:
>
>
> Aaron Wolfe wrote:
>> On 8/16/07, Marc Perkel <ma...@perkel.com> wrote:
>>
>>> OK - it's interesting that of all of you who responded this is the only
>>> person who is doing it right. I have to say that I'm somewhat surprised that
>>> so few people are preprocessing their email to reduce the SA load. As we all
>>> know SA is very processor and memory expensive.
>>>
>>
>> I think it's interesting that you somehow missed all the messages from
>> people who described how they *do* filter prior to SA. Considering
>> that you claim your setup never loses any mail, did you just forget to
>> read them somehow?
>>
>> Claiming that there is one right way to use SA is just silly. There
>> are so many different situations, and the right answer depends on the
>> amount of mail you process and the type of users you have. Sending
>> everything through SA might be a perfectly acceptable configuration
>> for a small domain that wants a single point of control and simple
>> configuration.
>>
>> What was the motivation behind your original post? What were you
>> hoping to learn?
>>
>>
>
> I was just curious based on some of the comments here that people really
> were running everything through SA. I'm just surprised that so many were
> doing that.
Why not do it? If you have one domain with three accounts what could it
hurt? There are a wide varety of users on this list.
We process around 120k messages a day after smtp checks and I think,
think, we sit about in the middle. I know there are larger users and I
know there are users with personal installs on SA as well.
To believe there is one best practice for using anything is painting
with a very large brush.
DAve
--
Three years now I've asked Google why they don't have a
logo change for Memorial Day. Why do they choose to do logos
for other non-international holidays, but nothing for
Veterans?
Maybe they forgot who made that choice possible.
Re: Question - How many of you run ALL your email through SA?
Posted by Marc Perkel <ma...@perkel.com>.
Aaron Wolfe wrote:
> On 8/16/07, Marc Perkel <ma...@perkel.com> wrote:
>
>> OK - it's interesting that of all of you who responded this is the only
>> person who is doing it right. I have to say that I'm somewhat surprised that
>> so few people are preprocessing their email to reduce the SA load. As we all
>> know SA is very processor and memory expensive.
>>
>
> I think it's interesting that you somehow missed all the messages from
> people who described how they *do* filter prior to SA. Considering
> that you claim your setup never loses any mail, did you just forget to
> read them somehow?
>
> Claiming that there is one right way to use SA is just silly. There
> are so many different situations, and the right answer depends on the
> amount of mail you process and the type of users you have. Sending
> everything through SA might be a perfectly acceptable configuration
> for a small domain that wants a single point of control and simple
> configuration.
>
> What was the motivation behind your original post? What were you
> hoping to learn?
>
>
I was just curious based on some of the comments here that people really
were running everything through SA. I'm just surprised that so many were
doing that.
Re: Question - How many of you run ALL your email through SA?
Posted by Aaron Wolfe <aa...@gmail.com>.
On 8/16/07, Marc Perkel <ma...@perkel.com> wrote:
>
> OK - it's interesting that of all of you who responded this is the only
> person who is doing it right. I have to say that I'm somewhat surprised that
> so few people are preprocessing their email to reduce the SA load. As we all
> know SA is very processor and memory expensive.
I think it's interesting that you somehow missed all the messages from
people who described how they *do* filter prior to SA. Considering
that you claim your setup never loses any mail, did you just forget to
read them somehow?
Claiming that there is one right way to use SA is just silly. There
are so many different situations, and the right answer depends on the
amount of mail you process and the type of users you have. Sending
everything through SA might be a perfectly acceptable configuration
for a small domain that wants a single point of control and simple
configuration.
What was the motivation behind your original post? What were you
hoping to learn?
>
> Personally, I'm filtering 1600 domains and I route less than 1% of incoming
> email through SA. SA does do a good job on the remaining 1% that I can't
> figure out with blacklists and whitelists and Exim tricks, but if I ran
> everything through SA I'd have to have a rack of dedicated SA servers.
>
> mailinglist@krausam.de wrote:
> Am Donnerstag, 16. August 2007 schrieb Marc Perkel:
>
>
> As opposed to preprocessing before using SA to reduce the load. (ie.
> using blacklist and whitelist before SA)
>
> I use:
>
> At rcpt time:
> callout to recipient
> zen.spamhaus.org <- Catches 90%
> bl.spamcop.net
> list.dsbl.org
> callout to sender
>
> At data time:
> clamd (malware is rejected)
> spamassassin (>10 Rejected, <10 add headers)
>
> I think i will lower the spamassassin scores to 8 in the near future.
>
> At the moment less then 5% spam reaches spamassasin.
>
>
>
RE: Question - How many of you run ALL your email through SA?
Posted by Skip Brott <sb...@dmp.com>.
> From: Marc Perkel [mailto:marc@perkel.com]
> OK - it's interesting that of all of you who responded this is the only
person who is doing it right.
I find this comment interesting because I don't agree with using spamhaus,
spamcop, or other similar services to determine whether mail should be
dropped/rejected. Systems can easily be errantly flagged - or temporarily
flagged - for unknown periods of time.
Our ISP provider had an extremely broad range of addresses blocked about a
year ago because of systems compromised on networks not belonging to our
company. For a period of several days, our company was effected - seeing
large numbers of bounces from systems rejecting because the range was
listed. This caused huge disruptions for our company, not to mention the
potential for significant losses of income.
If you were one of our customers expecting communication and are not
receiving replies for several days - are you blaming that on your own IT
department for using a blacklisting service? ...the actual comprimised
system? ...or my company?Customers don't want to hear that the problem is
someone else's. It becomes my problem.
That is just one of a handful of scenarios which have pursuaded me to
eliminate their use on my system. Unfortunately, I have no control over the
potential for the above situation repeating itself...
- Skip
Re: Question - How many of you run ALL your email through SA?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 16.08.07 09:39, Marc Perkel wrote:
> OK - it's interesting that of all of you who responded this is the only
> person who is doing it right.
Are you sure it's right? I do use similar configuration (and I plan use some
more filters, like greylisting on MX backups) but I wouldn't say so
generally it's right.
> I have to say that I'm somewhat surprised
> that so few people are preprocessing their email to reduce the SA load.
> As we all know SA is very processor and memory expensive.
the way you asked lead to thought that you are not interested in people who
do per-filtering, so I did not react.
> >Am Donnerstag, 16. August 2007 schrieb Marc Perkel:
> >>As opposed to preprocessing before using SA to reduce the load. (ie.
> >>using blacklist and whitelist before SA)
> mailinglist@krausam.de wrote:
> >I use:
> >
> >At rcpt time:
> >callout to recipient
> >zen.spamhaus.org <- Catches 90%
> >bl.spamcop.net
> >list.dsbl.org
> >callout to sender
> >
> >At data time:
> >clamd (malware is rejected)
> >spamassassin (>10 Rejected, <10 add headers)
> >
> >I think i will lower the spamassassin scores to 8 in the near future.
> >
> >At the moment less then 5% spam reaches spamassasin.
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
If Barbie is so popular, why do you have to buy her friends?
Re: Question - How many of you run ALL your email through SA?
Posted by Steven Kurylo <st...@aviawest.com>.
> I had great results from grey-listing but my users didn't like
> having to wait 30-60-90 minutes for mail, and I understand that. When
> you're on the phone with someone and they say "Just sent it," they
> expect you to have it in a matter of seconds. As I'm often in that
> positition, I had to support that view and remove the grey-list.
I use sa-exim, which means I can greylist after the message has been
scored. So I only greylist a message if it scores over 7 and toss
messages over 20.
This way the vast majority of messages are sent through instantly.
> I've tried aboslute RBL blocking, but I'm happier having RBL as a
> weighted factor counting for or against the spamminess of an email.
> We only process about 5,000 non-spam messages per day (out of about
> 45,000/day total) and are doing OK on a couple of old dual-processor
> systems running it through clamd and spamd with sendmail.
Same here. Sure its more cpu power, but it lowers false positives. CPU
cycles are cheap.
Re: Question - How many of you run ALL your email through SA?
Posted by Steven Stern <su...@sterndata.com>.
Marc Perkel wrote:
> OK - it's interesting that of all of you who responded this is the
> only person who is doing it right. I have to say that I'm somewhat
> surprised that so few people are preprocessing their email to reduce
> the SA load. As we all know SA is very processor and memory expensive.
>
> Personally, I'm filtering 1600 domains and I route less than 1% of
> incoming email through SA. SA does do a good job on the remaining 1%
> that I can't figure out with blacklists and whitelists and Exim
> tricks, but if I ran everything through SA I'd have to have a rack of
> dedicated SA servers.
>
> mailinglist@krausam.de wrote:
>> Am Donnerstag, 16. August 2007 schrieb Marc Perkel:
>>
>>> As opposed to preprocessing before using SA to reduce the load. (ie.
>>> using blacklist and whitelist before SA)
>>>
>>
>> I use:
>>
>> At rcpt time:
>> callout to recipient
>> zen.spamhaus.org <- Catches 90%
>> bl.spamcop.net
>> list.dsbl.org
>> callout to sender
>>
>> At data time:
>> clamd (malware is rejected)
>> spamassassin (>10 Rejected, <10 add headers)
>>
>> I think i will lower the spamassassin scores to 8 in the near future.
>>
>> At the moment less then 5% spam reaches spamassasin.
>>
>>
I had great results from grey-listing but my users didn't like
having to wait 30-60-90 minutes for mail, and I understand that. When
you're on the phone with someone and they say "Just sent it," they
expect you to have it in a matter of seconds. As I'm often in that
positition, I had to support that view and remove the grey-list. I've
tried aboslute RBL blocking, but I'm happier having RBL as a weighted
factor counting for or against the spamminess of an email. We only
process about 5,000 non-spam messages per day (out of about 45,000/day
total) and are doing OK on a couple of old dual-processor systems
running it through clamd and spamd with sendmail.
Re: Question - How many of you run ALL your email through SA?
Posted by Kai Schaetzl <ma...@conactive.com>.
Marc Perkel wrote on Thu, 16 Aug 2007 09:39:46 -0700:
> the only person who is doing it right.
Nonsense. You and I may think it's the right thing. But it's not the right
thing for everything. Just accept that.
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
Re: Question - How many of you run ALL your email through SA?
Posted by Chris Purves <ch...@northfolk.ca>.
Marc Perkel wrote:
> OK - it's interesting that of all of you who responded this is the only
> person who is doing it right.
I find it interesting that what seemed like a friendly question turned
out to be a quiz.
--
Chris Purves
"All science is either physics or stamp collecting." - Ernest Rutherford
Re: Question - How many of you run ALL your email through SA?
Posted by John Rudd <jr...@ucsc.edu>.
It's interesting to me that your chosen example of "doing it right" is
in fact doing 2 very wrong things (bl.spamcop.net as a blacklist, and
sender callback).
Further, I and several other people mentioned the same basic setups as
this one (minus the 2 mistakes I just mentioned), and I am in fact (and
said so) doing several things before SA specifically to reduce the load
of SA.
I usually think your messages start with well intentioned ideas that
just go off into wrong turns, but this one seems to indicate a more
fundamental problem in your thought processes.
Marc Perkel wrote:
> OK - it's interesting that of all of you who responded this is the only
> person who is doing it right. I have to say that I'm somewhat surprised
> that so few people are preprocessing their email to reduce the SA load.
> As we all know SA is very processor and memory expensive.
>
> Personally, I'm filtering 1600 domains and I route less than 1% of
> incoming email through SA. SA does do a good job on the remaining 1%
> that I can't figure out with blacklists and whitelists and Exim tricks,
> but if I ran everything through SA I'd have to have a rack of dedicated
> SA servers.
>
> mailinglist@krausam.de wrote:
>> Am Donnerstag, 16. August 2007 schrieb Marc Perkel:
>>
>>> As opposed to preprocessing before using SA to reduce the load. (ie.
>>> using blacklist and whitelist before SA)
>>>
>>
>> I use:
>>
>> At rcpt time:
>> callout to recipient
>> zen.spamhaus.org <- Catches 90%
>> bl.spamcop.net
>> list.dsbl.org
>> callout to sender
>>
>> At data time:
>> clamd (malware is rejected)
>> spamassassin (>10 Rejected, <10 add headers)
>> I think i will lower the spamassassin scores to 8 in the near future.
>>
>> At the moment less then 5% spam reaches spamassasin.
>>
>>
>
Re: Question - How many of you run ALL your email through SA?
Posted by Duane Hill <d....@yournetplus.com>.
On Tue, 21 Aug 2007 at 17:43 -0700, jrhett@netconsonance.com confabulated:
>
> On Aug 21, 2007, at 11:48 AM, Duane Hill wrote:
>> Ok. I just examined the clamav.pm plugin and it does appear to pass the
>> message text directly to the ClamAV daemon through the use of the
>> File::Scan::ClamAV perl module. Therefore, it doesn't sound like a temp
>> file is created.
>
> Read the code of that module.
The ClamAV plugin passes the test of the message using:
my ($code, $virus) = $clamav->streamscan(${$fulltext});
$fulltext is the text that was sent to the plugin from SA.
'streamscan' then establishes a TCP connection to the ClamAV daemon and
feeds the text to it:
sub streamscan {
my ($self) = shift;
my $data = join '', @_;
$self->_seterrstr;
my $conn = $self->_get_connection || return;
$self->_send($conn, "STREAM\n");
chomp(my $response = $conn->getline);
my @return;
if($response =~ /^PORT (\d+)/){
if((my $c = $self->_get_tcp_connection($1))){
$self->_send($c, $data);
$c->close;
chomp(my $r = $conn->getline);
if($r =~ /stream: (.+) FOUND/i){
@return = ('FOUND', $1);
} else {
@return = ('OK');
}
} else {
$conn->close;
return;
}
}
$conn->close;
return @return;
}
-------
_|_
(_| |
Re: Question - How many of you run ALL your email through SA?
Posted by Jo Rhett <jr...@netconsonance.com>.
On Aug 21, 2007, at 11:48 AM, Duane Hill wrote:
> Ok. I just examined the clamav.pm plugin and it does appear to pass
> the message text directly to the ClamAV daemon through the use of
> the File::Scan::ClamAV perl module. Therefore, it doesn't sound
> like a temp file is created.
Read the code of that module.
--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source
and other randomness
Re: Question - How many of you run ALL your email through SA?
Posted by Duane Hill <d....@yournetplus.com>.
On Tue, 21 Aug 2007 at 11:31 -0700, jrhett@netconsonance.com confabulated:
> On Aug 21, 2007, at 11:17 AM, Duane Hill wrote:
>> On Tue, 21 Aug 2007 at 11:03 -0700, jrhett@netconsonance.com confabulated:
>>> It seems to mostly help when it drops the message into a file for clamav
>>> to scan.
>>
>> Is that using the ClamAV plugin or outside of SA completely? I am currently
>> using the ClamAV plugin with the SaneSecurity additions.
>
> I'm using Amavisd, which invokes clamav itself before calling SA. So your
> environment may be entirely different.
>
> Examine the process and see if/when/how it uses temporary files. It always
> seems to be faster using a ramdisk for the temp files.
Ok. I just examined the clamav.pm plugin and it does appear to pass the
message text directly to the ClamAV daemon through the use of the
File::Scan::ClamAV perl module. Therefore, it doesn't sound like a temp
file is created.
-------
_|_
(_| |
Re: Question - How many of you run ALL your email through SA?
Posted by Jo Rhett <jr...@netconsonance.com>.
On Aug 21, 2007, at 1:42 PM, Marc Perkel wrote:
> I've been using Clam but I've heard of Amavisd - do I want it? What
> all does it do?
amavisd-new provides a nice front-end for virus and spamassassin
scanning. It's like using spamd, but a lot more featurefull. In my
case it was the easiest way to (a) snap into sendmail without using a
separate front-end scanner and (b) had useful end-user tools for
managing spam controls.
That said, it does white/black/etc listing in its own databases, not
the SA ones, etc etc. So research it.
--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source
and other randomness
Re: Question - How many of you run ALL your email through SA?
Posted by Bill Landry <bi...@inetmsg.com>.
Marc Perkel wrote:
>
>
> Jo Rhett wrote:
>> On Aug 21, 2007, at 11:17 AM, Duane Hill wrote:
>>> On Tue, 21 Aug 2007 at 11:03 -0700, jrhett@netconsonance.com
>>> confabulated:
>>>> It seems to mostly help when it drops the message into a file for
>>>> clamav to scan.
>>>
>>> Is that using the ClamAV plugin or outside of SA completely? I am
>>> currently using the ClamAV plugin with the SaneSecurity additions.
>>
>> I'm using Amavisd, which invokes clamav itself before calling SA. So
>> your environment may be entirely different.
>>
>> Examine the process and see if/when/how it uses temporary files. It
>> always seems to be faster using a ramdisk for the temp files.
>>
>
> I've been using Clam but I've heard of Amavisd - do I want it? What all
> does it do?
>
See: http://www.ijs.si/software/amavisd/ for details.
Bill
Re: Question - How many of you run ALL your email through SA?
Posted by Marc Perkel <ma...@perkel.com>.
Jo Rhett wrote:
> On Aug 21, 2007, at 11:17 AM, Duane Hill wrote:
>> On Tue, 21 Aug 2007 at 11:03 -0700, jrhett@netconsonance.com
>> confabulated:
>>> It seems to mostly help when it drops the message into a file for
>>> clamav to scan.
>>
>> Is that using the ClamAV plugin or outside of SA completely? I am
>> currently using the ClamAV plugin with the SaneSecurity additions.
>
> I'm using Amavisd, which invokes clamav itself before calling SA. So
> your environment may be entirely different.
>
> Examine the process and see if/when/how it uses temporary files. It
> always seems to be faster using a ramdisk for the temp files.
>
I've been using Clam but I've heard of Amavisd - do I want it? What all
does it do?
Re: Question - How many of you run ALL your email through SA?
Posted by Jo Rhett <jr...@netconsonance.com>.
On Aug 21, 2007, at 11:17 AM, Duane Hill wrote:
> On Tue, 21 Aug 2007 at 11:03 -0700, jrhett@netconsonance.com
> confabulated:
>> It seems to mostly help when it drops the message into a file for
>> clamav to scan.
>
> Is that using the ClamAV plugin or outside of SA completely? I am
> currently using the ClamAV plugin with the SaneSecurity additions.
I'm using Amavisd, which invokes clamav itself before calling SA. So
your environment may be entirely different.
Examine the process and see if/when/how it uses temporary files. It
always seems to be faster using a ramdisk for the temp files.
--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source
and other randomness
Re: Question - How many of you run ALL your email through SA?
Posted by Duane Hill <d....@yournetplus.com>.
On Tue, 21 Aug 2007 at 11:03 -0700, jrhett@netconsonance.com confabulated:
> On Aug 21, 2007, at 8:28 AM, Duane Hill wrote:
>> I have seen the suggestion recently in this thread to run SA from a ram
>> drive. I am going to experiment with that over the course of this next
>> weekend. I'm not quiet sure how much increase in speed I will get. All of
>> our userprefs, AWL and bayes are stored in MySQL tables.
>
> It seems to mostly help when it drops the message into a file for clamav to
> scan.
Is that using the ClamAV plugin or outside of SA completely? I am
currently using the ClamAV plugin with the SaneSecurity additions.
-------
_|_
(_| |
Re: Question - How many of you run ALL your email through SA?
Posted by Jo Rhett <jr...@netconsonance.com>.
On Aug 21, 2007, at 8:28 AM, Duane Hill wrote:
> I have seen the suggestion recently in this thread to run SA from a
> ram drive. I am going to experiment with that over the course of
> this next weekend. I'm not quiet sure how much increase in speed I
> will get. All of our userprefs, AWL and bayes are stored in MySQL
> tables.
It seems to mostly help when it drops the message into a file for
clamav to scan.
--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source
and other randomness
Re: Question - How many of you run ALL your email through SA?
Posted by Duane Hill <d....@yournetplus.com>.
On Tue, 21 Aug 2007 at 09:33 -0500, lm7812@gmail.com confabulated:
>> You're doing a LOT better than I am with it. Makes me wonder if I have
>> something set up wrong. My main SA server has a fast dual core Athlon
>> and 8 gigs of ram and it can get bogged down rather quickly. I wonder if
>> I'm doing something wrong ....
>
> Are you running 64bit OS? If so how stable are things? I thought
> about it to get by the 4Gbyte RAM limit but I chickened out since
> Directadmin lists 64bit support as beta.
Here we are running the amd64 FreeBSD v5.5 on a Dell PowerEdge 6850. It is
configured with 4x3ghz dual core zeon processors and 12 gig of ram. I have
increased the maximum data segment for the OS to 2 gig. It was defaulted
at 512 meg. I have set SA to use a max of 40 spamd children and keeping
20 spare laying around. Each child running takes up approximately 120 meg.
The server currently handles roughly 3.5 million messages every 24 hours.
I have seen the suggestion recently in this thread to run SA from a ram
drive. I am going to experiment with that over the course of this next
weekend. I'm not quiet sure how much increase in speed I will get. All of
our userprefs, AWL and bayes are stored in MySQL tables.
-------
_|_
(_| |
Re: Question - How many of you run ALL your email through SA?
Posted by Matt <lm...@gmail.com>.
> You're doing a LOT better than I am with it. Makes me wonder if I have
> something set up wrong. My main SA server has a fast dual core Athlon
> and 8 gigs of ram and it can get bogged down rather quickly. I wonder if
> I'm doing something wrong ....
Are you running 64bit OS? If so how stable are things? I thought
about it to get by the 4Gbyte RAM limit but I chickened out since
Directadmin lists 64bit support as beta.
I have a Directadmin server which is basically Apache, Exim, Dovecot
with a nice GUI. I added Spamassassin which now filters about 2000
email accounts. Its recently upgraded to a dual core AMD CPU with
4Gbyte of DDR2 and SATA2 drive. Handles things much better then the
old Socket A 2800+ with 2Gbyte of RAM and PATA drive.
I do not use any blacklists at MTA time except sbl.spamhaus.org. When
I ran others I had to many custommers complain about blocking there
email. This server works in an ISP setting so must deal with 1500+
unique custommers who have differing ideas on how there email should
be filtered. I just add headers and knock priority on hits to low so
they can filter easilly with OE or there email client of choice.
Matt
Re: Question - How many of you run ALL your email through SA?
Posted by Jo Rhett <jr...@netconsonance.com>.
On Aug 19, 2007, at 7:22 AM, Marc Perkel wrote:
> You're doing a LOT better than I am with it. Makes me wonder if I
> have something set up wrong. My main SA server has a fast dual core
> Athlon and 8 gigs of ram and it can get bogged down rather quickly.
> I wonder if I'm doing something wrong ....
If you have the memory, configure SA to use a ramdisk instead of
local disk. That's good for an 8x increase. It does limit incoming
message size to ramdisk size, but that's fine for our environment.
FWIW: I'm using amavisd-new via milter from sendmail. I dunno if
you're using something more IO-intensive.
--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source
and other randomness
Re: Question - How many of you run ALL your email through SA?
Posted by James Lay <jl...@slave-tothe-box.net>.
On 8/19/07 8:22 AM, "Marc Perkel" <ma...@perkel.com> wrote:
>
>
> Jo Rhett wrote:
>> Marc Perkel wrote:
>>> OK - it's interesting that of all of you who responded this is the
>>> only person who is doing it right. I have to say that I'm somewhat
>>> surprised that so few people are preprocessing their email to reduce
>>> the SA load. As we all know SA is very processor and memory expensive.
>>>
>>> Personally, I'm filtering 1600 domains and I route less than 1% of
>>> incoming email through SA. SA does do a good job on the remaining 1%
>>> that I can't figure out with blacklists and whitelists and Exim
>>> tricks, but if I ran everything through SA I'd have to have a rack of
>>> dedicated SA servers.
>>
>> I accept that you feel that what are you doing is "right", but I
>> personally find the convenience and power/control of SA to be very
>> useful. I'm routing about 1100 domains with 120k messages per hour on
>> a fairly basic Athlon system no problem, so I'm not quite sure why you
>> find SA to be so intensive.
>>
>> (note that I'm using fairly stock SA with lots of SARE rulesets but no
>> plugins to speak of)
>>
>
> You're doing a LOT better than I am with it. Makes me wonder if I have
> something set up wrong. My main SA server has a fast dual core Athlon
> and 8 gigs of ram and it can get bogged down rather quickly. I wonder if
> I'm doing something wrong ....
>
For what it's worth:
File messages : from Aug 1 00:02:40 to Aug 19 08:29:28
Total number of emails processed by the spam filter : 47950
Number of spams : 43770 ( 91.28%)
Number of clean messages : 4180 ( 8.72%)
Average message analysis time : 4.94 seconds
Average spam analysis time : 4.54 seconds
Average clean message analysis time : 8.75 seconds
Average message score : 19.52
Average spam score : 22.40
Average clean message score : -7.68
Total spam volume : 267 Mbytes
Total clean volume : 87 Mbytes
I've been able to handle a 5 second lagtime on email quite nicely :)
James
Re: Question - How many of you run ALL your email through SA?
Posted by Marc Perkel <ma...@perkel.com>.
Jo Rhett wrote:
> Marc Perkel wrote:
>> OK - it's interesting that of all of you who responded this is the
>> only person who is doing it right. I have to say that I'm somewhat
>> surprised that so few people are preprocessing their email to reduce
>> the SA load. As we all know SA is very processor and memory expensive.
>>
>> Personally, I'm filtering 1600 domains and I route less than 1% of
>> incoming email through SA. SA does do a good job on the remaining 1%
>> that I can't figure out with blacklists and whitelists and Exim
>> tricks, but if I ran everything through SA I'd have to have a rack of
>> dedicated SA servers.
>
> I accept that you feel that what are you doing is "right", but I
> personally find the convenience and power/control of SA to be very
> useful. I'm routing about 1100 domains with 120k messages per hour on
> a fairly basic Athlon system no problem, so I'm not quite sure why you
> find SA to be so intensive.
>
> (note that I'm using fairly stock SA with lots of SARE rulesets but no
> plugins to speak of)
>
You're doing a LOT better than I am with it. Makes me wonder if I have
something set up wrong. My main SA server has a fast dual core Athlon
and 8 gigs of ram and it can get bogged down rather quickly. I wonder if
I'm doing something wrong ....
Re: Question - How many of you run ALL your email through SA?
Posted by Jo Rhett <jr...@netconsonance.com>.
Marc Perkel wrote:
> OK - it's interesting that of all of you who responded this is the only
> person who is doing it right. I have to say that I'm somewhat surprised
> that so few people are preprocessing their email to reduce the SA load.
> As we all know SA is very processor and memory expensive.
>
> Personally, I'm filtering 1600 domains and I route less than 1% of
> incoming email through SA. SA does do a good job on the remaining 1%
> that I can't figure out with blacklists and whitelists and Exim tricks,
> but if I ran everything through SA I'd have to have a rack of dedicated
> SA servers.
I accept that you feel that what are you doing is "right", but I
personally find the convenience and power/control of SA to be very
useful. I'm routing about 1100 domains with 120k messages per hour on a
fairly basic Athlon system no problem, so I'm not quite sure why you
find SA to be so intensive.
(note that I'm using fairly stock SA with lots of SARE rulesets but no
plugins to speak of)
Re: Question - How many of you run ALL your email through SA?
Posted by Marc Perkel <ma...@perkel.com>.
Eric A. Hall wrote:
> On 8/16/2007 12:39 PM, Marc Perkel wrote:
>
>> OK - it's interesting that of all of you who responded this is the only
>> person who is doing it right. I have to say that I'm somewhat surprised
>> that so few people are preprocessing their email to reduce the SA load.
>> As we all know SA is very processor and memory expensive.
>>
>> Personally, I'm filtering 1600 domains and I route less than 1% of
>> incoming email through SA. SA does do a good job on the remaining 1%
>> that I can't figure out with blacklists and whitelists and Exim tricks,
>> but if I ran everything through SA I'd have to have a rack of dedicated
>> SA servers.
>>
>
> third-party blacklists are good indicators but they are not perfectly
> accurate. the errors make them unsuitable as a sole metric, but are by
> definition very good inputs for spamassassin's probability scoring systems.
>
> for those of us that can afford this approach it works very well. I'm
> sorry you can't, but that's not our fault.
>
>
I have a few blacklists that I trust but one thing I do is that I have a
big white list of good hosts that let me route more than half og my good
email around SA which reduces load and increases accuracy.
Re: Question - How many of you run ALL your email through SA?
Posted by Marc Perkel <ma...@perkel.com>.
Eric A. Hall wrote:
> On 8/16/2007 12:39 PM, Marc Perkel wrote:
>
>> OK - it's interesting that of all of you who responded this is the only
>> person who is doing it right. I have to say that I'm somewhat surprised
>> that so few people are preprocessing their email to reduce the SA load.
>> As we all know SA is very processor and memory expensive.
>>
>> Personally, I'm filtering 1600 domains and I route less than 1% of
>> incoming email through SA. SA does do a good job on the remaining 1%
>> that I can't figure out with blacklists and whitelists and Exim tricks,
>> but if I ran everything through SA I'd have to have a rack of dedicated
>> SA servers.
>>
>
> third-party blacklists are good indicators but they are not perfectly
> accurate. the errors make them unsuitable as a sole metric, but are by
> definition very good inputs for spamassassin's probability scoring systems.
>
> for those of us that can afford this approach it works very well. I'm
> sorry you can't, but that's not our fault.
>
>
What I do to avoid problems with blacklists is I use whitelists and what
I call yellow lists to avoid false positives on blacklists. Whitelist
allow me to route most ham around SA as well reducing load and
increasing accuracy.
Re: Question - How many of you run ALL your email through SA?
Posted by Jon Trulson <jo...@radscan.com>.
On Mon, 20 Aug 2007, David B Funk wrote:
> On Mon, 20 Aug 2007, Duane Hill wrote:
>
>> On Mon, 20 Aug 2007 at 16:24 -0600, jon@radscan.com confabulated:
>>
> [snip..]
>>> I have to second that... In the early days when spammers were just
>>> getting started, we started using some RBL's at the MTA level. ORBS
>>> was one I believe. Then they went away and started tagging
>>> everything as spam, and of course we started rejecting everything.
>>>
>>> Lesson learned - we will not depend on any external RBL as an
>>> absolute pass/fail test ever again :) We use greylisting on the
>>> secondary MX's, but everything goes through SA eventually before
>>> entering our internal mail system. Works great.
>>
>> Most blacklists I know of that have gone away in the past set DNS to
>> return 127.0.0.2 to ALL requests that came in. Most of the email lists I'm
>> on received posts by other list members with reguards to the list going
>> away. I would speculate that was the reason your messages started tagging
>> as spam.
>>
>> One such list I remember was ordb.org.
>
> ordb.org RIP 12/31/2006
> dorkslayers.com RIP 9/15/2003
> osirusoft.com RIP 8/20/2003
> orbz.org RIP 3/25/2002
> orbs.org RIP 6/3/2001
>
> And that's just from this millenium. ;)
>
> Returning FP to ALL requests is the fastest way to wake up brain-damaged
> sites that don't get the clue.
ordb.org, osirusoft.com, orbs.org - those were ones we used IIRC.
Guess we didn't have a clue then. As mentioned earlier, for our
setup anyway, it is unwise to pin pass/fail on RBL's. They can be
wrong, or go away.
--
Jon Trulson
mailto:jon@radscan.com
#include <std/disclaimer.h>
"No Kill I" -Horta
Re: Question - How many of you run ALL your email through SA?
Posted by David B Funk <db...@engineering.uiowa.edu>.
On Mon, 20 Aug 2007, Duane Hill wrote:
> On Mon, 20 Aug 2007 at 16:24 -0600, jon@radscan.com confabulated:
>
[snip..]
> > I have to second that... In the early days when spammers were just
> > getting started, we started using some RBL's at the MTA level. ORBS
> > was one I believe. Then they went away and started tagging
> > everything as spam, and of course we started rejecting everything.
> >
> > Lesson learned - we will not depend on any external RBL as an
> > absolute pass/fail test ever again :) We use greylisting on the
> > secondary MX's, but everything goes through SA eventually before
> > entering our internal mail system. Works great.
>
> Most blacklists I know of that have gone away in the past set DNS to
> return 127.0.0.2 to ALL requests that came in. Most of the email lists I'm
> on received posts by other list members with reguards to the list going
> away. I would speculate that was the reason your messages started tagging
> as spam.
>
> One such list I remember was ordb.org.
ordb.org RIP 12/31/2006
dorkslayers.com RIP 9/15/2003
osirusoft.com RIP 8/20/2003
orbz.org RIP 3/25/2002
orbs.org RIP 6/3/2001
And that's just from this millenium. ;)
Returning FP to ALL requests is the fastest way to wake up brain-damaged
sites that don't get the clue.
--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{
Re: Question - How many of you run ALL your email through SA?
Posted by Jon Trulson <jo...@radscan.com>.
On Mon, 20 Aug 2007, Duane Hill wrote:
> On Mon, 20 Aug 2007 at 16:24 -0600, jon@radscan.com confabulated:
>
>> On Fri, 17 Aug 2007, Eric A. Hall wrote:
>>
>>>
>>> On 8/16/2007 12:39 PM, Marc Perkel wrote:
>>>> OK - it's interesting that of all of you who responded this is the only
>>>> person who is doing it right. I have to say that I'm somewhat surprised
[...]
>
> Most blacklists I know of that have gone away in the past set DNS to return
> 127.0.0.2 to ALL requests that came in. Most of the email lists I'm on
> received posts by other list members with reguards to the list going away. I
> would speculate that was the reason your messages started tagging as spam.
>
> One such list I remember was ordb.org.
>
Yes, ordb. Knew it was something like that. It may be true that
they posted something to a list - unfortunately, I was not
subscribed.
Nonetheless, we won't do that again.
--
Jon Trulson
mailto:jon@radscan.com
#include <std/disclaimer.h>
"No Kill I" -Horta
Re: Question - How many of you run ALL your email through SA?
Posted by Duane Hill <d....@yournetplus.com>.
On Mon, 20 Aug 2007 at 16:24 -0600, jon@radscan.com confabulated:
> On Fri, 17 Aug 2007, Eric A. Hall wrote:
>
>>
>> On 8/16/2007 12:39 PM, Marc Perkel wrote:
>>> OK - it's interesting that of all of you who responded this is the only
>>> person who is doing it right. I have to say that I'm somewhat surprised
>>> that so few people are preprocessing their email to reduce the SA load.
>>> As we all know SA is very processor and memory expensive.
>>>
>>> Personally, I'm filtering 1600 domains and I route less than 1% of
>>> incoming email through SA. SA does do a good job on the remaining 1%
>>> that I can't figure out with blacklists and whitelists and Exim tricks,
>>> but if I ran everything through SA I'd have to have a rack of dedicated
>>> SA servers.
>>
>> third-party blacklists are good indicators but they are not perfectly
>> accurate. the errors make them unsuitable as a sole metric, but are by
>> definition very good inputs for spamassassin's probability scoring systems.
>>
>> for those of us that can afford this approach it works very well. I'm
>> sorry you can't, but that's not our fault.
>>
>
> I have to second that... In the early days when spammers were just
> getting started, we started using some RBL's at the MTA level. ORBS
> was one I believe. Then they went away and started tagging
> everything as spam, and of course we started rejecting everything.
>
> Lesson learned - we will not depend on any external RBL as an
> absolute pass/fail test ever again :) We use greylisting on the
> secondary MX's, but everything goes through SA eventually before
> entering our internal mail system. Works great.
Most blacklists I know of that have gone away in the past set DNS to
return 127.0.0.2 to ALL requests that came in. Most of the email lists I'm
on received posts by other list members with reguards to the list going
away. I would speculate that was the reason your messages started tagging
as spam.
One such list I remember was ordb.org.
-------
_|_
(_| |
Re: Question - How many of you run ALL your email through SA?
Posted by Jon Trulson <jo...@radscan.com>.
On Fri, 17 Aug 2007, Eric A. Hall wrote:
>
> On 8/16/2007 12:39 PM, Marc Perkel wrote:
>> OK - it's interesting that of all of you who responded this is the only
>> person who is doing it right. I have to say that I'm somewhat surprised
>> that so few people are preprocessing their email to reduce the SA load.
>> As we all know SA is very processor and memory expensive.
>>
>> Personally, I'm filtering 1600 domains and I route less than 1% of
>> incoming email through SA. SA does do a good job on the remaining 1%
>> that I can't figure out with blacklists and whitelists and Exim tricks,
>> but if I ran everything through SA I'd have to have a rack of dedicated
>> SA servers.
>
> third-party blacklists are good indicators but they are not perfectly
> accurate. the errors make them unsuitable as a sole metric, but are by
> definition very good inputs for spamassassin's probability scoring systems.
>
> for those of us that can afford this approach it works very well. I'm
> sorry you can't, but that's not our fault.
>
I have to second that... In the early days when spammers were just
getting started, we started using some RBL's at the MTA level. ORBS
was one I believe. Then they went away and started tagging
everything as spam, and of course we started rejecting everything.
Lesson learned - we will not depend on any external RBL as an
absolute pass/fail test ever again :) We use greylisting on the
secondary MX's, but everything goes through SA eventually before
entering our internal mail system. Works great.
--
Jon Trulson
mailto:jon@radscan.com
#include <std/disclaimer.h>
"No Kill I" -Horta
Re: Question - How many of you run ALL your email through SA?
Posted by "Eric A. Hall" <eh...@ehsco.com>.
On 8/16/2007 12:39 PM, Marc Perkel wrote:
> OK - it's interesting that of all of you who responded this is the only
> person who is doing it right. I have to say that I'm somewhat surprised
> that so few people are preprocessing their email to reduce the SA load.
> As we all know SA is very processor and memory expensive.
>
> Personally, I'm filtering 1600 domains and I route less than 1% of
> incoming email through SA. SA does do a good job on the remaining 1%
> that I can't figure out with blacklists and whitelists and Exim tricks,
> but if I ran everything through SA I'd have to have a rack of dedicated
> SA servers.
third-party blacklists are good indicators but they are not perfectly
accurate. the errors make them unsuitable as a sole metric, but are by
definition very good inputs for spamassassin's probability scoring systems.
for those of us that can afford this approach it works very well. I'm
sorry you can't, but that's not our fault.
--
Eric A. Hall http://www.ehsco.com/
Internet Core Protocols http://www.oreilly.com/catalog/coreprot/
Re: Question - How many of you run ALL your email through SA?
Posted by Marc Perkel <ma...@perkel.com>.
OK - it's interesting that of all of you who responded this is the only
person who is doing it right. I have to say that I'm somewhat surprised
that so few people are preprocessing their email to reduce the SA load.
As we all know SA is very processor and memory expensive.
Personally, I'm filtering 1600 domains and I route less than 1% of
incoming email through SA. SA does do a good job on the remaining 1%
that I can't figure out with blacklists and whitelists and Exim tricks,
but if I ran everything through SA I'd have to have a rack of dedicated
SA servers.
mailinglist@krausam.de wrote:
> Am Donnerstag, 16. August 2007 schrieb Marc Perkel:
>
>> As opposed to preprocessing before using SA to reduce the load. (ie.
>> using blacklist and whitelist before SA)
>>
>
> I use:
>
> At rcpt time:
> callout to recipient
> zen.spamhaus.org <- Catches 90%
> bl.spamcop.net
> list.dsbl.org
> callout to sender
>
> At data time:
> clamd (malware is rejected)
> spamassassin (>10 Rejected, <10 add headers)
>
> I think i will lower the spamassassin scores to 8 in the near future.
>
> At the moment less then 5% spam reaches spamassasin.
>
>
Re: Question - How many of you run ALL your email through SA?
Posted by ma...@krausam.de.
Am Donnerstag, 16. August 2007 schrieb Marc Perkel:
> As opposed to preprocessing before using SA to reduce the load. (ie.
> using blacklist and whitelist before SA)
I use:
At rcpt time:
callout to recipient
zen.spamhaus.org <- Catches 90%
bl.spamcop.net
list.dsbl.org
callout to sender
At data time:
clamd (malware is rejected)
spamassassin (>10 Rejected, <10 add headers)
I think i will lower the spamassassin scores to 8 in the near future.
At the moment less then 5% spam reaches spamassasin.
--
Micha Krause
Jabber: SMS-King@jabber.org
Email: Micha@krausam.de
Re: Question - How many of you run ALL your email through SA?
Posted by Brian Godette <bg...@idcomm.com>.
Marc Perkel wrote:
> As opposed to preprocessing before using SA to reduce the load. (ie.
> using blacklist and whitelist before SA)
>
We don't.
We use a locally modified MaRBL that uses weighted scoring, RHSBLs
against helo/sender domain/reverse, and the BOTNET plugin (each
meta-rule gets its own weight), then greylisting (gld policy server),
then clamav w/sane+msrbl, then finally SA. All this does for us is
reduce the load on the spamd servers and bayes database, the amount of
marked spam that would actually get to a user that /dev/null's over a
certain score does not change significantly.
This brings the detected spam rate to about 2% of all delivery attempts
or 14.8% of what SA sees; what the user sees may be much less depending
on what they set their /dev/null score to.
We used to use just greylisting, but it was becoming far less effective
over time (~8 months ago), by adding weighted rbl lookups to reject at
SMTP time and then greylist the rest, the amount of spam as seen by SA
dropped to 12% of what it was with just greylisting alone.
At some point we should add in SPF checks to MaRBL and maybe integrate
p0f from its latest release.
RE: Question - How many of you run ALL your email through SA?
Posted by Rob Sterenborg <R....@netsourcing.nl>.
Marc Perkel wrote:
> As opposed to preprocessing before using SA to reduce the load. (ie.
> using blacklist and whitelist before SA)
We don't. I could sum it up but it basically looks like John's setup,
except we also use SPF and greylisting in the SMTP session and that
ClamAV isn't the only virusscanner.
RE: Question - How many of you run ALL your email through SA?
Posted by Ben Spencer <be...@moody.edu>.
We do not allow SA to process everything.
Internet -> Grey Listing -> Link checks (milter-link from SnertSoft) ->
Virus Checking -> SA -> Internal Mail Server
Due to the grey listing, this allows SA to see only a small percentage of
email which really is really being attempted (and from what we see, that
non-retries are not misconfigured MTAs). This has allowed us to keep our
infrastructure fairly small (only 1 SA box with 1 CPU) while dealing with
about 250K email / day delivery attampts.
benji
---
Benji Spencer
System Administrator
Ph: 312-329-2288
> -----Original Message-----
> From: Marc Perkel [mailto:marc@perkel.com]
> Sent: Wednesday, August 15, 2007 10:11 PM
> To: users@spamassassin.apache.org
> Subject: Question - How many of you run ALL your email through SA?
>
> As opposed to preprocessing before using SA to reduce the load. (ie.
> using blacklist and whitelist before SA)
>