You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Henrique Fernandes <sf...@gmail.com> on 2010/06/22 16:52:15 UTC
TMPDIR as a tmpfs
It is safe to use spamassassin tmpdir on a tmpfs mounted system ?
And if its safe it would have a better performance ?
Here where i work we have big problems with the hard drives, because we
basically are sharing virtual machines disk over nfs. and spamassasin is a
virtual machine.
Any other tips for better performance ?
[]'sf.rique
Re: TMPDIR as a tmpfs
Posted by RW <rw...@googlemail.com>.
On Tue, 22 Jun 2010 19:47:00 -0700
Gary Smith <ga...@holdstead.com> wrote:
> > I had a very dramatic performance improvement by switching bayes and
> > awl
> > databases to MySQL instead of the default BerkeleyDB. It costs more
> > RAM, CPU, and disk space, but scan times reduced dramatically. I'm
> > certain we were I/O bound before this change because we had plenty
> > of RAM and CPU available.
> >
>
> I agree to the bayes DB being MySQL. When we switched to that years
> ago it was night and day.
It'd be interesting to see the effect of bayes_learn_to_journal on gdb.
No-one ever mentions trying it, but it might be faster than MySQL.
AFAIK there's no equivalent for AWL, so that would have to be
turned-off to see the benefit.
Re: TMPDIR as a tmpfs
Posted by Henrique Fernandes <sf...@gmail.com>.
it is taking about 3 seconds each email, i have pyzor and dcc, i am already
runing with mysql db.
After i get the statistis i will post here if tmpfs is faster or not! I made
some script that can't see anyway for losing email!
Thanks for all advise!
And sorry about my english
[]'sf.rique
On Tue, Jun 22, 2010 at 11:47 PM, Gary Smith <ga...@holdstead.com>wrote:
> > I don't know if it is safe. I suspect it will function normally, but I
> > think you'd be in danger of losing a few messages on an unexpected
> > reboot.
> >
> > I had a very dramatic performance improvement by switching bayes and
> > awl
> > databases to MySQL instead of the default BerkeleyDB. It costs more
> > RAM, CPU, and disk space, but scan times reduced dramatically. I'm
> > certain we were I/O bound before this change because we had plenty of
> > RAM and CPU available.
> >
>
> I agree to the bayes DB being MySQL. When we switched to that years ago it
> was night and day. We have a central MySQL cluster feeding multiple SA
> instances without any problem. Generally we are running VM's for SA as we
> can randomly spin then up when we need them on machines with idle CPU's.
>
> Gary
>
RE: TMPDIR as a tmpfs
Posted by Gary Smith <ga...@holdstead.com>.
> I don't know if it is safe. I suspect it will function normally, but I
> think you'd be in danger of losing a few messages on an unexpected
> reboot.
>
> I had a very dramatic performance improvement by switching bayes and
> awl
> databases to MySQL instead of the default BerkeleyDB. It costs more
> RAM, CPU, and disk space, but scan times reduced dramatically. I'm
> certain we were I/O bound before this change because we had plenty of
> RAM and CPU available.
>
I agree to the bayes DB being MySQL. When we switched to that years ago it was night and day. We have a central MySQL cluster feeding multiple SA instances without any problem. Generally we are running VM's for SA as we can randomly spin then up when we need them on machines with idle CPU's.
Gary
Re: TMPDIR as a tmpfs
Posted by Adam Moffett <ad...@plexicomm.net>.
I don't know if it is safe. I suspect it will function normally, but I
think you'd be in danger of losing a few messages on an unexpected reboot.
I had a very dramatic performance improvement by switching bayes and awl
databases to MySQL instead of the default BerkeleyDB. It costs more
RAM, CPU, and disk space, but scan times reduced dramatically. I'm
certain we were I/O bound before this change because we had plenty of
RAM and CPU available.
> It is safe to use spamassassin tmpdir on a tmpfs mounted system ?
>
> And if its safe it would have a better performance ?
>
> Here where i work we have big problems with the hard drives, because
> we basically are sharing virtual machines disk over nfs. and
> spamassasin is a virtual machine.
>
> Any other tips for better performance ?
>
>
>
>
> []'sf.rique
Re: TMPDIR as a tmpfs
Posted by Henrique Fernandes <sf...@gmail.com>.
pyzor and dcc
Só it might be it ?
If i am not using wont get in tmp ?
Right now i have 2 server with spamassassin, one i just put everything in
spamassassin in tmpfs and the other one i did not change anything
[]'sf.rique
On Tue, Jun 22, 2010 at 12:46 PM, Gary Smith <ga...@holdstead.com>wrote:
>
>
> My ram dos not get full, i do not have so many process, i limit it in
> postfix.
>
> It reduces the chances of losing emails if i do not have many process of
> spamassassin runing.
>
> So is safe or not to use tmpfs for tempdir in spamassassin. ?
>
> This way, everything that spamassassin have to do with the message it does
> on tmpfs.
>
> ---
>
> Outside smtp -> postfix -> postfix after queue -> spamassassin -> postfix
> -> destination
>
> Even if SA is going slow, no email will be lost. If it is, something else
> is broken. If postfix and SA are on the same VM, it's the postfix queue
> that could be slowing things down. In this case there isn't much you can do
> as you need postfix to be on persistent media.
>
> Also, my understanding is that SA only uses temp files for Razor and DCC
> checks. Otherwise it should be in ram anyway. Are you doing Razor or DCC?
>
RE: TMPDIR as a tmpfs
Posted by Gary Smith <ga...@holdstead.com>.
My ram dos not get full, i do not have so many process, i limit it in postfix.
It reduces the chances of losing emails if i do not have many process of spamassassin runing.
So is safe or not to use tmpfs for tempdir in spamassassin. ?
This way, everything that spamassassin have to do with the message it does on tmpfs.
---
Outside smtp -> postfix -> postfix after queue -> spamassassin -> postfix -> destination
Even if SA is going slow, no email will be lost. If it is, something else is broken. If postfix and SA are on the same VM, it's the postfix queue that could be slowing things down. In this case there isn't much you can do as you need postfix to be on persistent media.
Also, my understanding is that SA only uses temp files for Razor and DCC checks. Otherwise it should be in ram anyway. Are you doing Razor or DCC?
Re: TMPDIR as a tmpfs
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 22.06.10 12:40, Henrique Fernandes wrote:
> My ram dos not get full, i do not have so many process, i limit it in
> postfix.
>
> It reduces the chances of losing emails if i do not have many process of
> spamassassin runing.
>
> So is safe or not to use tmpfs for tempdir in spamassassin. ?
I use tmpfs for /tmp for years. I don't remember any problems with it.
And it increases system performance in many cases.
Note that /tmp is for TEMPORARY storage, that doesn't need to retain after
reboot.
> This way, everything that spamassassin have to do with the message it does
> on tmpfs.
does SA use /tmp at all?
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
99 percent of lawyers give the rest a bad name.
Re: TMPDIR as a tmpfs
Posted by Henrique Fernandes <sf...@gmail.com>.
My ram dos not get full, i do not have so many process, i limit it in
postfix.
It reduces the chances of losing emails if i do not have many process of
spamassassin runing.
So is safe or not to use tmpfs for tempdir in spamassassin. ?
This way, everything that spamassassin have to do with the message it does
on tmpfs.
[]'sf.rique
On Tue, Jun 22, 2010 at 12:28 PM, Gary Smith <ga...@holdstead.com>wrote:
> > It is safe to use spamassassin tmpdir on a tmpfs mounted system ?
> > And if its safe it would have a better performance ?
> > Here where i work we have big problems with the hard drives, because we
> basically are sharing virtual machines disk over nfs. and spamassasin is a
> virtual machine.
> > Any other tips for better performance ?
>
> Ram, lots of it. This seem to have the biggest impact on my systems. The
> only time I see disk thrashing on our SA and ClamAV VM's is when they are
> starved for ram.
>
RE: TMPDIR as a tmpfs
Posted by Gary Smith <ga...@holdstead.com>.
> It is safe to use spamassassin tmpdir on a tmpfs mounted system ?
> And if its safe it would have a better performance ?
> Here where i work we have big problems with the hard drives, because we basically are sharing virtual machines disk over nfs. and spamassasin is a virtual machine.
> Any other tips for better performance ?
Ram, lots of it. This seem to have the biggest impact on my systems. The only time I see disk thrashing on our SA and ClamAV VM's is when they are starved for ram.
Re: TMPDIR as a tmpfs
Posted by Eduardo Casarero <ec...@gmail.com>.
2010/6/22 Henrique Fernandes <sf...@gmail.com>
> It is safe to use spamassassin tmpdir on a tmpfs mounted system ?
>
> And if its safe it would have a better performance ?
>
> Here where i work we have big problems with the hard drives, because we
> basically are sharing virtual machines disk over nfs. and spamassasin is a
> virtual machine.
>
> Any other tips for better performance ?
>
>
>
>
> []'sf.rique
>
Re: TMPDIR as a tmpfs
Posted by RW <rw...@googlemail.com>.
On Sat, 26 Jun 2010 16:33:32 -0700
"jdow" <jd...@earthlink.net> wrote:
> > Potentially. If you've got memory free for it, it certainly
> > shouldn't perform worse.
>
> That might be a big if with a huge downside, Daryl.
>
> If the memory used by tmpfs forces SpamAssassin into memory swapping
> any speed advantages are more than merely wiped out, aren't they?
No, because, at worst, you are exchanging one type of disk access for
another.
A tmpfs partition is memory-backed by swap, a file on a normal
filesystem is cached in memory. There's not all that much difference,
they are both memory backed by a physical disk backing-store. A well
designed kernel will place the physical memory where it's most
effective, whether that's caching a file or keeping the tmpfs or
process pages in ram.
The advantage of tmpfs is not that it's stored in memory, it's that the
kernel can put-off updating the backing store indefinitely - a
temporary file can be created, updated and deleted without troubling
the hard drive.
Re: TMPDIR as a tmpfs
Posted by jdow <jd...@earthlink.net>.
From: "Daryl C. W. O'Shea" <sp...@dostech.ca>
Sent: Saturday, 2010/June/26 15:23
> On 22/06/2010 10:52 AM, Henrique Fernandes wrote:
>> It is safe to use spamassassin tmpdir on a tmpfs mounted system ?
>
> Yes it's safe.
>
>> And if its safe it would have a better performance ?
>
> Potentially. If you've got memory free for it, it certainly shouldn't
> perform worse.
That might be a big if with a huge downside, Daryl.
If the memory used by tmpfs forces SpamAssassin into memory swapping
any speed advantages are more than merely wiped out, aren't they?
{o.o} <- her number 1 SA rule is to avoid swap at all costs.
Re: TMPDIR as a tmpfs
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 22/06/2010 10:52 AM, Henrique Fernandes wrote:
> It is safe to use spamassassin tmpdir on a tmpfs mounted system ?
Yes it's safe.
> And if its safe it would have a better performance ?
Potentially. If you've got memory free for it, it certainly shouldn't
perform worse.
Daryl