You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Ed Russell <ed...@hardsoftsystems.com> on 2006/02/10 17:51:04 UTC
General assistance
If everyone would indulge me I would like to put forth the setup I am
utilizing and get some feedback. I have a box that I have been using for
some time which acts as a pass-through filter for many domains (currently
about 100) for spam, this is a fairly high traffic server processing about
150,000 to 200,000 messages per day. I use the following method.
Based upon a redhat 6.2 box running kernel 2.2.26, PIV with 2 Gigs of RAM.
Qmail runs which accepts the email from the world (with a
concurrencyincoming of 100) and passes it through qmail-scanner (which calls
spamd) and spamassassin which checks the email and writes spam status to the
header. Each message gets then passed through a procmail filter which will
delete it if it is spam. The procmail filter is:
:0
* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
{
EXITCODE=99
:0
/dev/null
}
:0
* ^X-Spam-Status: Yes
{
EXITCODE=99
:0
/dev/null
}
:0
* ^^rom[ ]
{
LOG="*** Dropped F off From_ header! Fixing up. "
:0 fhw
| sed -e '1s/^/F/'
}
:0
/dev/null
Mail that is clean gets passed off to a second qmail install which then
delivers the mail to our servers using smtproutes.
This has been working fine for a few years now, but recently we have
experienced major delays in the processing of email. Due to the very high
volume pretty much all the time the system is handling 100 concurrent
incoming pieces of email. Of course with everything else going on it is not
uncommon for this system to have up to 400 processes running. Sometimes
mail can take hours to get through to its destination. What I have
discovered is that if I disable RBL checks and razor, pyzor and dcc the
delay goes away. However, the effectiveness of the filter reduces.
Am I completely off base in the way I have this all setup? I have went with
a higher speed HD to increase the threshold on file I/O. Can I tune the
performance of razor etc while maintaining delivery time? Is there anything
else I should be considering? If I have not explained things well or more
information is needed I will certainly provide anything.
Thanks
---------------------------------------------------
Talk is cheap since supply always exceeds demand.
---------------------------------------------------
RE: General assistance
Posted by Ed Russell <ed...@hardsoftsystems.com>.
You are completely correct, qmail-scanner does use spamc to talk to the
already running spamd. I just had trouble explaining what the setup was....
I may indeed look into having procmail be the agent for Spamassassin. As
for automatic deletion, well that's a decision we made and for the most part
it works. We just ensure that we are not too aggressive on the rules.
Ed
-----Original Message-----
From: jdow [mailto:jdow@earthlink.net]
Sent: Saturday, February 11, 2006 12:28 AM
To: users@spamassassin.apache.org
Subject: Re: General assistance
No, Ed, qmail-scanner should not initiate spamd. It should use spamc to
call the already running spamd. I hope that is what you mean. That is
what stood the hairs on end. It made me wonder if you really knew what
was going on. {o.o}
And seriously, if you are using procmail it's perhaps better to fire
off spamc from procmail. That way you can skip SA scanning for some
specific addresses, if you want. Or you can skip SA scanning if the
message size is too big. If you're running procmail anyway it might as
well be the agent for running SpamAssassin. That way you are SURE the
markups are there for when you delegate the spam to /dev/null.
And as a general rule I believe dumping mail to /dev/null is asking for
"I sent you the ebay notifications you needed! I can't help it if your
spam filter deleted them! Why'd you give me a bad review, you EW$#@ing
3-)(#$*&&&!"
{o.o}
----- Original Message -----
From: "Ed Russell" <ed...@hardsoftsystems.com>
To: <us...@spamassassin.apache.org>
Sent: 2006 February, 10, Friday 20:11
Subject: RE: General assistance
>I think you are confused as to how I have set this up. Qmail-scanner is my
> replacement qmail queue. Qmail simply receives mail from the outside
world,
> then passes it to qmail-scanner for processing. Qmail-scanner initiates
> spamd which scans the mail and off it goes. From there procmail will look
> at the mail and determine if the spam status is marked in the header, if
yes
> it kills the mail, if not it passes it along. Keep in mind no users
> whatsoever live on this box. It is as I mentioned a pass through filter.
>
>
>
> -----Original Message-----
> From: jdow [mailto:jdow@earthlink.net]
> Sent: Friday, February 10, 2006 10:55 PM
> To: users@spamassassin.apache.org
> Subject: Re: General assistance
>
> From: "Ed Russell" <ed...@hardsoftsystems.com>
>
>> If everyone would indulge me I would like to put forth the setup I am
>> utilizing and get some feedback. I have a box that I have been using
for
>> some time which acts as a pass-through filter for many domains (currently
>> about 100) for spam, this is a fairly high traffic server processing
about
>> 150,000 to 200,000 messages per day. I use the following method.
>>
>> Based upon a redhat 6.2 box running kernel 2.2.26, PIV with 2 Gigs of
RAM.
>>
>> Qmail runs which accepts the email from the world (with a
>> concurrencyincoming of 100) and passes it through qmail-scanner (which
> calls
>> spamd) and spamassassin which checks the email and writes spam status to
> the
>> header. Each message gets then passed through a procmail filter which
> will
>> delete it if it is spam. The procmail filter is:
>
> I note the other answers and thought I'd comment because the above
> description of your mail topology raised the hairs on the back of my
> neck. (And that takes doing considering their length. {^_-})
>
> First I not you say Qmail (it's own punishment) feets qmail-scanner.
> The qmail-scanner calls spamd? Naw, can't DO that. AND it calls
> spamassassin? That's even stranger. But then it goes to procmail for
> the delivery.
>
> My topology is somewhat different but useful. If you are using
qmail-scanner
> only to make the spamassassin run and the procmail run then jettison it
> and go to procmail directly. That MAY reduce the machine load a little.
> Also make sure spamd is running, exactly once, from your /etc/init.d
> files or the equivalent on BSDs. You'd then use spamc to get to the
> SpamAssassin run. You show some data below. (I am not sure what the
> EXITCODE is supposed to do for you. I never set it here. But that may
> be because I use procmail alone. It exits and mail is "delivered" either
> to a diversion directory, /dev/null, or the user's mailbox.)
>
> Anyway, you can call spamc from inside procmail this way:
>
> :0
> * < 500000
> * !^List-Id: .*(spamassassin\.apache.\org)
> | /usr/bin/spamc -t 150 -u $USER
>
>> :0
>> * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
>> {
>> EXITCODE=99
>> :0
>> /dev/null
>> }
>>
>> :0
>> * ^X-Spam-Status: Yes
>> {
>> EXITCODE=99
>> :0
>> /dev/null
>> }
>>
>> :0
>> * ^^rom[ ]
>> {
>> LOG="*** Dropped F off From_ header! Fixing up. "
>>
>> :0 fhw
>> | sed -e '1s/^/F/'
>> }
>>
>> :0
>> /dev/null
>>
>> Mail that is clean gets passed off to a second qmail install which then
>> delivers the mail to our servers using smtproutes.
>
> Ouch. And what is that final redirect of EVERYTHING to /dev/null? I just
> let procmail deliver it.
>
> {o.o}
>
>
Re: General assistance
Posted by jdow <jd...@earthlink.net>.
No, Ed, qmail-scanner should not initiate spamd. It should use spamc to
call the already running spamd. I hope that is what you mean. That is
what stood the hairs on end. It made me wonder if you really knew what
was going on. {o.o}
And seriously, if you are using procmail it's perhaps better to fire
off spamc from procmail. That way you can skip SA scanning for some
specific addresses, if you want. Or you can skip SA scanning if the
message size is too big. If you're running procmail anyway it might as
well be the agent for running SpamAssassin. That way you are SURE the
markups are there for when you delegate the spam to /dev/null.
And as a general rule I believe dumping mail to /dev/null is asking for
"I sent you the ebay notifications you needed! I can't help it if your
spam filter deleted them! Why'd you give me a bad review, you EW$#@ing
3-)(#$*&&&!"
{o.o}
----- Original Message -----
From: "Ed Russell" <ed...@hardsoftsystems.com>
To: <us...@spamassassin.apache.org>
Sent: 2006 February, 10, Friday 20:11
Subject: RE: General assistance
>I think you are confused as to how I have set this up. Qmail-scanner is my
> replacement qmail queue. Qmail simply receives mail from the outside world,
> then passes it to qmail-scanner for processing. Qmail-scanner initiates
> spamd which scans the mail and off it goes. From there procmail will look
> at the mail and determine if the spam status is marked in the header, if yes
> it kills the mail, if not it passes it along. Keep in mind no users
> whatsoever live on this box. It is as I mentioned a pass through filter.
>
>
>
> -----Original Message-----
> From: jdow [mailto:jdow@earthlink.net]
> Sent: Friday, February 10, 2006 10:55 PM
> To: users@spamassassin.apache.org
> Subject: Re: General assistance
>
> From: "Ed Russell" <ed...@hardsoftsystems.com>
>
>> If everyone would indulge me I would like to put forth the setup I am
>> utilizing and get some feedback. I have a box that I have been using for
>> some time which acts as a pass-through filter for many domains (currently
>> about 100) for spam, this is a fairly high traffic server processing about
>> 150,000 to 200,000 messages per day. I use the following method.
>>
>> Based upon a redhat 6.2 box running kernel 2.2.26, PIV with 2 Gigs of RAM.
>>
>> Qmail runs which accepts the email from the world (with a
>> concurrencyincoming of 100) and passes it through qmail-scanner (which
> calls
>> spamd) and spamassassin which checks the email and writes spam status to
> the
>> header. Each message gets then passed through a procmail filter which
> will
>> delete it if it is spam. The procmail filter is:
>
> I note the other answers and thought I'd comment because the above
> description of your mail topology raised the hairs on the back of my
> neck. (And that takes doing considering their length. {^_-})
>
> First I not you say Qmail (it's own punishment) feets qmail-scanner.
> The qmail-scanner calls spamd? Naw, can't DO that. AND it calls
> spamassassin? That's even stranger. But then it goes to procmail for
> the delivery.
>
> My topology is somewhat different but useful. If you are using qmail-scanner
> only to make the spamassassin run and the procmail run then jettison it
> and go to procmail directly. That MAY reduce the machine load a little.
> Also make sure spamd is running, exactly once, from your /etc/init.d
> files or the equivalent on BSDs. You'd then use spamc to get to the
> SpamAssassin run. You show some data below. (I am not sure what the
> EXITCODE is supposed to do for you. I never set it here. But that may
> be because I use procmail alone. It exits and mail is "delivered" either
> to a diversion directory, /dev/null, or the user's mailbox.)
>
> Anyway, you can call spamc from inside procmail this way:
>
> :0
> * < 500000
> * !^List-Id: .*(spamassassin\.apache.\org)
> | /usr/bin/spamc -t 150 -u $USER
>
>> :0
>> * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
>> {
>> EXITCODE=99
>> :0
>> /dev/null
>> }
>>
>> :0
>> * ^X-Spam-Status: Yes
>> {
>> EXITCODE=99
>> :0
>> /dev/null
>> }
>>
>> :0
>> * ^^rom[ ]
>> {
>> LOG="*** Dropped F off From_ header! Fixing up. "
>>
>> :0 fhw
>> | sed -e '1s/^/F/'
>> }
>>
>> :0
>> /dev/null
>>
>> Mail that is clean gets passed off to a second qmail install which then
>> delivers the mail to our servers using smtproutes.
>
> Ouch. And what is that final redirect of EVERYTHING to /dev/null? I just
> let procmail deliver it.
>
> {o.o}
>
>
RE: General assistance
Posted by Ed Russell <ed...@hardsoftsystems.com>.
I think you are confused as to how I have set this up. Qmail-scanner is my
replacement qmail queue. Qmail simply receives mail from the outside world,
then passes it to qmail-scanner for processing. Qmail-scanner initiates
spamd which scans the mail and off it goes. From there procmail will look
at the mail and determine if the spam status is marked in the header, if yes
it kills the mail, if not it passes it along. Keep in mind no users
whatsoever live on this box. It is as I mentioned a pass through filter.
-----Original Message-----
From: jdow [mailto:jdow@earthlink.net]
Sent: Friday, February 10, 2006 10:55 PM
To: users@spamassassin.apache.org
Subject: Re: General assistance
From: "Ed Russell" <ed...@hardsoftsystems.com>
> If everyone would indulge me I would like to put forth the setup I am
> utilizing and get some feedback. I have a box that I have been using for
> some time which acts as a pass-through filter for many domains (currently
> about 100) for spam, this is a fairly high traffic server processing about
> 150,000 to 200,000 messages per day. I use the following method.
>
> Based upon a redhat 6.2 box running kernel 2.2.26, PIV with 2 Gigs of RAM.
>
> Qmail runs which accepts the email from the world (with a
> concurrencyincoming of 100) and passes it through qmail-scanner (which
calls
> spamd) and spamassassin which checks the email and writes spam status to
the
> header. Each message gets then passed through a procmail filter which
will
> delete it if it is spam. The procmail filter is:
I note the other answers and thought I'd comment because the above
description of your mail topology raised the hairs on the back of my
neck. (And that takes doing considering their length. {^_-})
First I not you say Qmail (it's own punishment) feets qmail-scanner.
The qmail-scanner calls spamd? Naw, can't DO that. AND it calls
spamassassin? That's even stranger. But then it goes to procmail for
the delivery.
My topology is somewhat different but useful. If you are using qmail-scanner
only to make the spamassassin run and the procmail run then jettison it
and go to procmail directly. That MAY reduce the machine load a little.
Also make sure spamd is running, exactly once, from your /etc/init.d
files or the equivalent on BSDs. You'd then use spamc to get to the
SpamAssassin run. You show some data below. (I am not sure what the
EXITCODE is supposed to do for you. I never set it here. But that may
be because I use procmail alone. It exits and mail is "delivered" either
to a diversion directory, /dev/null, or the user's mailbox.)
Anyway, you can call spamc from inside procmail this way:
:0
* < 500000
* !^List-Id: .*(spamassassin\.apache.\org)
| /usr/bin/spamc -t 150 -u $USER
> :0
> * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
> {
> EXITCODE=99
> :0
> /dev/null
> }
>
> :0
> * ^X-Spam-Status: Yes
> {
> EXITCODE=99
> :0
> /dev/null
> }
>
> :0
> * ^^rom[ ]
> {
> LOG="*** Dropped F off From_ header! Fixing up. "
>
> :0 fhw
> | sed -e '1s/^/F/'
> }
>
> :0
> /dev/null
>
> Mail that is clean gets passed off to a second qmail install which then
> delivers the mail to our servers using smtproutes.
Ouch. And what is that final redirect of EVERYTHING to /dev/null? I just
let procmail deliver it.
{o.o}
Re: General assistance
Posted by jdow <jd...@earthlink.net>.
From: "Ed Russell" <ed...@hardsoftsystems.com>
> If everyone would indulge me I would like to put forth the setup I am
> utilizing and get some feedback. I have a box that I have been using for
> some time which acts as a pass-through filter for many domains (currently
> about 100) for spam, this is a fairly high traffic server processing about
> 150,000 to 200,000 messages per day. I use the following method.
>
> Based upon a redhat 6.2 box running kernel 2.2.26, PIV with 2 Gigs of RAM.
>
> Qmail runs which accepts the email from the world (with a
> concurrencyincoming of 100) and passes it through qmail-scanner (which calls
> spamd) and spamassassin which checks the email and writes spam status to the
> header. Each message gets then passed through a procmail filter which will
> delete it if it is spam. The procmail filter is:
I note the other answers and thought I'd comment because the above
description of your mail topology raised the hairs on the back of my
neck. (And that takes doing considering their length. {^_-})
First I not you say Qmail (it's own punishment) feets qmail-scanner.
The qmail-scanner calls spamd? Naw, can't DO that. AND it calls
spamassassin? That's even stranger. But then it goes to procmail for
the delivery.
My topology is somewhat different but useful. If you are using qmail-scanner
only to make the spamassassin run and the procmail run then jettison it
and go to procmail directly. That MAY reduce the machine load a little.
Also make sure spamd is running, exactly once, from your /etc/init.d
files or the equivalent on BSDs. You'd then use spamc to get to the
SpamAssassin run. You show some data below. (I am not sure what the
EXITCODE is supposed to do for you. I never set it here. But that may
be because I use procmail alone. It exits and mail is "delivered" either
to a diversion directory, /dev/null, or the user's mailbox.)
Anyway, you can call spamc from inside procmail this way:
:0
* < 500000
* !^List-Id: .*(spamassassin\.apache.\org)
| /usr/bin/spamc -t 150 -u $USER
> :0
> * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
> {
> EXITCODE=99
> :0
> /dev/null
> }
>
> :0
> * ^X-Spam-Status: Yes
> {
> EXITCODE=99
> :0
> /dev/null
> }
>
> :0
> * ^^rom[ ]
> {
> LOG="*** Dropped F off From_ header! Fixing up. "
>
> :0 fhw
> | sed -e '1s/^/F/'
> }
>
> :0
> /dev/null
>
> Mail that is clean gets passed off to a second qmail install which then
> delivers the mail to our servers using smtproutes.
Ouch. And what is that final redirect of EVERYTHING to /dev/null? I just
let procmail deliver it.
{o.o}
Re: General assistance
Posted by DAve <da...@pixelhammer.com>.
Joey wrote:
> Dave,
>
> What paramters are you using for logging with the caching name server?
> I currently use this:
>
> logging {
> category lame-servers { null; };
> };
>
> Thanks,
>
> Joey
>
I was speaking of dnscache, the program, not dnscache as in "a cacheing
DNS server". See http://cr.yp.to/djbdns.html. It can log in a very
verbose way, generating gigabytes of log files a day.
What you have above for Bind.
DAve
>
> -----Original Message-----
> From: DAve [mailto:dave.list@pixelhammer.com]
> Sent: Friday, February 10, 2006 12:28 PM
> To: users@spamassassin.apache.org
> Subject: Re: General assistance
>
> Ed Russell wrote:
>
>>If everyone would indulge me I would like to put forth the setup I am
>>utilizing and get some feedback. I have a box that I have been using for
>>some time which acts as a pass-through filter for many domains
>>(currently about 100) for spam, this is a fairly high traffic server
>>processing about 150,000 to 200,000 messages per day. I use the following
>
> method.
>
>>Based upon a redhat 6.2 box running kernel 2.2.26, PIV with 2 Gigs of RAM.
>>
>>Qmail runs which accepts the email from the world (with a
>>concurrencyincoming of 100) and passes it through qmail-scanner (which
>>calls
>>spamd) and spamassassin which checks the email and writes spam status
>>to the header. Each message gets then passed through a procmail
>>filter which will delete it if it is spam. The procmail filter is:
>>
>>:0
>>* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\* {
>> EXITCODE=99
>> :0
>> /dev/null
>>}
>>
>>:0
>>* ^X-Spam-Status: Yes
>>{
>> EXITCODE=99
>> :0
>> /dev/null
>>}
>>
>>:0
>>* ^^rom[ ]
>>{
>> LOG="*** Dropped F off From_ header! Fixing up. "
>>
>> :0 fhw
>> | sed -e '1s/^/F/'
>>}
>>
>>:0
>>/dev/null
>>
>>Mail that is clean gets passed off to a second qmail install which
>>then delivers the mail to our servers using smtproutes.
>>
>>This has been working fine for a few years now, but recently we have
>>experienced major delays in the processing of email. Due to the very
>>high volume pretty much all the time the system is handling 100
>>concurrent incoming pieces of email. Of course with everything else
>>going on it is not uncommon for this system to have up to 400
>>processes running. Sometimes mail can take hours to get through to
>>its destination. What I have discovered is that if I disable RBL
>>checks and razor, pyzor and dcc the delay goes away. However, the
>
> effectiveness of the filter reduces.
>
>>Am I completely off base in the way I have this all setup? I have
>>went with a higher speed HD to increase the threshold on file I/O.
>>Can I tune the performance of razor etc while maintaining delivery
>>time? Is there anything else I should be considering? If I have not
>>explained things well or more information is needed I will certainly
>
> provide anything.
>
>>Thanks
>
>
> Since you are running qmail, consider doing your rbl checks in qmail-smtpd.
> No sense scanning a message if you can drop it at the door first.
>
> Also, are your running dnscache? I run dnscache on all my servers, web,
> webmail, toasters, etc. It can speed things up considerably as it will cache
> your RBL lookups, SURBL lookups, etc. It's a nice thing to do for the URIBL
> and SURBL folks too.
>
> If you do run dnscache, consider turning logging off once you are configured
> and satisfied it works as intended. dnscache can keep a disk pretty busy
> with it's potential to log a lot of data.
>
> DAve
>
>
>
>
RE: General assistance
Posted by Joey <Jo...@Web56.Net>.
Dave,
What paramters are you using for logging with the caching name server?
I currently use this:
logging {
category lame-servers { null; };
};
Thanks,
Joey
-----Original Message-----
From: DAve [mailto:dave.list@pixelhammer.com]
Sent: Friday, February 10, 2006 12:28 PM
To: users@spamassassin.apache.org
Subject: Re: General assistance
Ed Russell wrote:
> If everyone would indulge me I would like to put forth the setup I am
> utilizing and get some feedback. I have a box that I have been using for
> some time which acts as a pass-through filter for many domains
> (currently about 100) for spam, this is a fairly high traffic server
> processing about 150,000 to 200,000 messages per day. I use the following
method.
>
> Based upon a redhat 6.2 box running kernel 2.2.26, PIV with 2 Gigs of RAM.
>
> Qmail runs which accepts the email from the world (with a
> concurrencyincoming of 100) and passes it through qmail-scanner (which
> calls
> spamd) and spamassassin which checks the email and writes spam status
> to the header. Each message gets then passed through a procmail
> filter which will delete it if it is spam. The procmail filter is:
>
> :0
> * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\* {
> EXITCODE=99
> :0
> /dev/null
> }
>
> :0
> * ^X-Spam-Status: Yes
> {
> EXITCODE=99
> :0
> /dev/null
> }
>
> :0
> * ^^rom[ ]
> {
> LOG="*** Dropped F off From_ header! Fixing up. "
>
> :0 fhw
> | sed -e '1s/^/F/'
> }
>
> :0
> /dev/null
>
> Mail that is clean gets passed off to a second qmail install which
> then delivers the mail to our servers using smtproutes.
>
> This has been working fine for a few years now, but recently we have
> experienced major delays in the processing of email. Due to the very
> high volume pretty much all the time the system is handling 100
> concurrent incoming pieces of email. Of course with everything else
> going on it is not uncommon for this system to have up to 400
> processes running. Sometimes mail can take hours to get through to
> its destination. What I have discovered is that if I disable RBL
> checks and razor, pyzor and dcc the delay goes away. However, the
effectiveness of the filter reduces.
>
> Am I completely off base in the way I have this all setup? I have
> went with a higher speed HD to increase the threshold on file I/O.
> Can I tune the performance of razor etc while maintaining delivery
> time? Is there anything else I should be considering? If I have not
> explained things well or more information is needed I will certainly
provide anything.
>
> Thanks
Since you are running qmail, consider doing your rbl checks in qmail-smtpd.
No sense scanning a message if you can drop it at the door first.
Also, are your running dnscache? I run dnscache on all my servers, web,
webmail, toasters, etc. It can speed things up considerably as it will cache
your RBL lookups, SURBL lookups, etc. It's a nice thing to do for the URIBL
and SURBL folks too.
If you do run dnscache, consider turning logging off once you are configured
and satisfied it works as intended. dnscache can keep a disk pretty busy
with it's potential to log a lot of data.
DAve
Re: General assistance
Posted by DAve <da...@pixelhammer.com>.
Ed Russell wrote:
> If everyone would indulge me I would like to put forth the setup I am
> utilizing and get some feedback. I have a box that I have been using for
> some time which acts as a pass-through filter for many domains (currently
> about 100) for spam, this is a fairly high traffic server processing about
> 150,000 to 200,000 messages per day. I use the following method.
>
> Based upon a redhat 6.2 box running kernel 2.2.26, PIV with 2 Gigs of RAM.
>
> Qmail runs which accepts the email from the world (with a
> concurrencyincoming of 100) and passes it through qmail-scanner (which calls
> spamd) and spamassassin which checks the email and writes spam status to the
> header. Each message gets then passed through a procmail filter which will
> delete it if it is spam. The procmail filter is:
>
> :0
> * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
> {
> EXITCODE=99
> :0
> /dev/null
> }
>
> :0
> * ^X-Spam-Status: Yes
> {
> EXITCODE=99
> :0
> /dev/null
> }
>
> :0
> * ^^rom[ ]
> {
> LOG="*** Dropped F off From_ header! Fixing up. "
>
> :0 fhw
> | sed -e '1s/^/F/'
> }
>
> :0
> /dev/null
>
> Mail that is clean gets passed off to a second qmail install which then
> delivers the mail to our servers using smtproutes.
>
> This has been working fine for a few years now, but recently we have
> experienced major delays in the processing of email. Due to the very high
> volume pretty much all the time the system is handling 100 concurrent
> incoming pieces of email. Of course with everything else going on it is not
> uncommon for this system to have up to 400 processes running. Sometimes
> mail can take hours to get through to its destination. What I have
> discovered is that if I disable RBL checks and razor, pyzor and dcc the
> delay goes away. However, the effectiveness of the filter reduces.
>
> Am I completely off base in the way I have this all setup? I have went with
> a higher speed HD to increase the threshold on file I/O. Can I tune the
> performance of razor etc while maintaining delivery time? Is there anything
> else I should be considering? If I have not explained things well or more
> information is needed I will certainly provide anything.
>
> Thanks
Since you are running qmail, consider doing your rbl checks in
qmail-smtpd. No sense scanning a message if you can drop it at the door
first.
Also, are your running dnscache? I run dnscache on all my servers, web,
webmail, toasters, etc. It can speed things up considerably as it will
cache your RBL lookups, SURBL lookups, etc. It's a nice thing to do for
the URIBL and SURBL folks too.
If you do run dnscache, consider turning logging off once you are
configured and satisfied it works as intended. dnscache can keep a disk
pretty busy with it's potential to log a lot of data.
DAve