You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Res <re...@ausics.net> on 2009/04/15 00:56:22 UTC
sa-compile
Is there a method of speeding this beast up? I can build four entire
kernels and their modules from scratch in the same time it takes this
thing to compile (45mins) is it really worth using this method?
Last time I tried without it, I noticed next to no difference, opinions?
--
Res
-Beware of programmers who carry screwdrivers
Re: sa-compile
Posted by Res <re...@ausics.net>.
On Wed, 15 Apr 2009, Rick Macdougall wrote:
> jp wrote:
>> It only takes a minute or three on my systems depending on load. 21 seconds
>> on a zero load dual core virtual machine with 2gb ram.
>>
>> On Wed, Apr 15, 2009 at 08:56:22AM +1000, Res wrote:
>>> Is there a method of speeding this beast up? I can build four entire
>>> kernels and their modules from scratch in the same time it takes this
>>> thing to compile (45mins) is it really worth using this method?
>>> Last time I tried without it, I noticed next to no difference, opinions?
>>>
>
> Normal sa-update && sa-compile takes about 2 minutes here.
>
> If I add JM's saught rules it takes over 30 minutes.
>
> Just FYI.
Rick, et al,
You've hit the nail on the head...
Apr 16 00:03:31 mx0-3 sa-update: start
Apr 16 00:04:08 mx0-3 sa-update: compiling
Apr 16 00:52:27 mx0-3 sa-update: completed
... remove sought rules, and...
Apr 16 10:44:09 mx0-3 sa-update: start
Apr 16 10:44:25 mx0-3 sa-update: compiling
Apr 16 10:52:45 mx0-3 sa-update: completed
sought is no longer sought'n :)
Thanks for the heads up, much appreciated
--
Res
-Beware of programmers who carry screwdrivers
Re: sa-compile
Posted by Raymond Dijkxhoorn <ra...@prolocation.net>.
Hi!
>> Normal sa-update && sa-compile takes about 2 minutes here.
>> If I add JM's saught rules it takes over 30 minutes.
> Here's another data point. With JM's sought and sought-fraud rules the
> compile takes less than 7 minutes on a server running an Intel Core 2 Duo
> running at 2.13 GHz.
>
> # time sa-compile
> real 6m46.466s
> user 6m34.792s
> sys 0m9.251s
Same here. compile times went up like crazy with the sought rules.
JM, is this normal?
Bye,
Raymond.
Re: sa-compile
Posted by Justin Mason <jm...@jmason.org>.
A lot of stuff is cached between runs in ~/.spamassassin...
--j.
On Thu, Apr 16, 2009 at 02:56, Larry Nedry <sp...@bluestreak.net> wrote:
> On 4/15/09 at 10:30 AM -0400 Rick Macdougall wrote:
>>Normal sa-update && sa-compile takes about 2 minutes here.
>>If I add JM's saught rules it takes over 30 minutes.
>
> Here's another data point. With JM's sought and sought-fraud rules the
> compile takes less than 7 minutes on a server running an Intel Core 2 Duo
> running at 2.13 GHz.
>
> # time sa-compile
> real 6m46.466s
> user 6m34.792s
> sys 0m9.251s
>
> Nedry
>
>
Re: sa-compile
Posted by Michael Scheidell <sc...@secnap.net>.
well.. thats not it. im using 0.13.5 also.
Larry Nedry wrote:
> On 4/16/09 at 7:44 AM -0400 Michael Scheidell wrote:
>
>> Larry: what version of re2c are you using?
>>
>
> re2c 0.13.5
>
--
Michael Scheidell, CTO
Phone: 561-999-5000, x 1259
> *| *SECNAP Network Security Corporation
* Certified SNORT Integrator
* 2008-9 Hot Company Award Winner, World Executive Alliance
* Five-Star Partner Program 2009, VARBusiness
* Best Anti-Spam Product 2008, Network Products Guide
* King of Spam Filters, SC Magazine 2008
_________________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
_________________________________________________________________________
Re: sa-compile
Posted by Larry Nedry <sp...@bluestreak.net>.
On 4/16/09 at 7:44 AM -0400 Michael Scheidell wrote:
>Larry: what version of re2c are you using?
re2c 0.13.5
Re: sa-compile
Posted by Michael Scheidell <sc...@secnap.net>.
> On 4/15/09 at 10:30 AM -0400 Rick Macdougall wrote:
>> Normal sa-update && sa-compile takes about 2 minutes here.
>> If I add JM's saught rules it takes over 30 minutes.
>
> Here's another data point. With JM's sought and sought-fraud rules the
> compile takes less than 7 minutes on a server running an Intel Core 2 Duo
> running at 2.13 GHz.
>
> # time sa-compile
> real 6m46.466s
> user 6m34.792s
> sys 0m9.251s
>
I suspect that it might have to do also with the version or re2c.
I think when we hit .13 it got worse.
Larry: what version of re2c are you using?
(and, yes, my compiles of sought rules take 30 mins, ) if you crontab this
once an hour, you really only have 30 mins of coverage. So I do it once a
day.
--
Michael Scheidell, CTO
>|SECNAP Network Security
Finalist 2009 Network Products Guide Hot Companies
FreeBSD SpamAssassin Ports maintainer
_________________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
_________________________________________________________________________
Re: sa-compile
Posted by Larry Nedry <sp...@bluestreak.net>.
On 4/15/09 at 10:30 AM -0400 Rick Macdougall wrote:
>Normal sa-update && sa-compile takes about 2 minutes here.
>If I add JM's saught rules it takes over 30 minutes.
Here's another data point. With JM's sought and sought-fraud rules the
compile takes less than 7 minutes on a server running an Intel Core 2 Duo
running at 2.13 GHz.
# time sa-compile
real 6m46.466s
user 6m34.792s
sys 0m9.251s
Nedry
Re: sa-compile
Posted by Rick Macdougall <ri...@ummm-beer.com>.
jp wrote:
> It only takes a minute or three on my systems depending on load. 21
> seconds on a zero load dual core virtual machine with 2gb ram.
>
> On Wed, Apr 15, 2009 at 08:56:22AM +1000, Res wrote:
>> Is there a method of speeding this beast up? I can build four entire
>> kernels and their modules from scratch in the same time it takes this thing
>> to compile (45mins) is it really worth using this method?
>> Last time I tried without it, I noticed next to no difference, opinions?
>>
>>
>> --
>> Res
>>
>> -Beware of programmers who carry screwdrivers
>
Normal sa-update && sa-compile takes about 2 minutes here.
If I add JM's saught rules it takes over 30 minutes.
Just FYI.
Regards,
Rick
Re: sa-compile
Posted by jp <jp...@saucer.midcoast.com>.
It only takes a minute or three on my systems depending on load. 21
seconds on a zero load dual core virtual machine with 2gb ram.
On Wed, Apr 15, 2009 at 08:56:22AM +1000, Res wrote:
> Is there a method of speeding this beast up? I can build four entire
> kernels and their modules from scratch in the same time it takes this thing
> to compile (45mins) is it really worth using this method?
> Last time I tried without it, I noticed next to no difference, opinions?
>
>
> --
> Res
>
> -Beware of programmers who carry screwdrivers
--
/*
Jason Philbrook | Midcoast Internet Solutions - Wireless and DSL
KB1IOJ | Broadband Internet Access, Dialup, and Hosting
http://f64.nu/ | for Midcoast Maine http://www.midcoast.com/
*/
RE: sa-compile
Posted by RobertH <ro...@abbacomm.net>.
for those in the know re the programming and speed of processing using
sa-compile...
it appears the fules compile fast without the sought ruleset applied.
and time to compile increases by roughly (very rough) a factor of 10 with
sought ruleset applied.
is that time extra time spent strictly in the cpu and ram cycles or are
their other slower factors involved?
is the processing by sa-compile serial or can it be paralleled out to many
processors at the same time?
sorry, i dont know and havent looked at the code.
what i am wondering is how much processing speed or how many processors does
one have to throw at it to get it down "substantially" and then equate that
to a new server box in costs so to speak
:-)
- rh
Re: sa-compile
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Wed, 2009-04-15 at 17:31 -0400, Michael Scheidell wrote:
> > Is there a method of speeding this beast up? I can build four entire
> > kernels and their modules from scratch in the same time it takes this
> > thing to compile (45mins) is it really worth using this method?
> > Last time I tried without it, I noticed next to no difference, opinions?
>
> Same here, sought rules take an extra 30mins (normally is 2mins)
Wonder why that is -- due to the excessively long metas? The sub-rules'
REs are quite trivial.
Would constructing it using a binary (or n-ary, with small upper bound
of n) tree speed the compilation up?
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: sa-compile
Posted by Michael Scheidell <sc...@secnap.net>.
> Is there a method of speeding this beast up? I can build four entire
> kernels and their modules from scratch in the same time it takes this
> thing to compile (45mins) is it really worth using this method?
> Last time I tried without it, I noticed next to no difference, opinions?
>
Same here, sought rules take an extra 30mins (normally is 2mins)
--
Michael Scheidell, CTO
>|SECNAP Network Security
Finalist 2009 Network Products Guide Hot Companies
FreeBSD SpamAssassin Ports maintainer
_________________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
_________________________________________________________________________