You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Hamad Ali <cr...@hotmail.com> on 2011/03/11 21:05:47 UTC
how to disable network tests?
hi folks --- everything seems working like chicken. I'm loving SA so far.
However, I would like to disable all network tests (each mail takes ~10 seconds!). Except that I dunno how to do it the neat way.
Will the tests be disabled if their score is 0? I know that would lead into disabling the effect of a rule on the decision making of SA (i.e. Spam/Ham marking), but would SA exclude them from running too?
I need to disable all BLs, DNS queries, and anything that uses the internet. Kindly advise.
Thank you guys -- May OOP Raise and Shine!H
Re: how to disable network tests?
Posted by Benny Pedersen <me...@junc.org>.
On Sat, 12 Mar 2011 00:05:47 +0400, Hamad Ali <cr...@hotmail.com>
wrote:
> However, I would like to disable all network tests (each mail takes ~10
> seconds!). Except that I dunno how to do it the neat way.
disable plugins that do the hard work of testing dnsbl is plain simple in
*.pre files
spamassassin 2>&1 -D --lint | less
what plugins do you want to disable ?
see what are loaded, and disable what is not usefull to your needs
but:
disable to much in spamassassin can make it not so usefull any more to use
it will turn over to not be a bennefit to call it at all
and lastly nameservers in /etc/resolv.conf should work
test like this:
dig +norecurse @ip-of-nameserver gmail.com mx
if its time out then ip-of-nameserver does not work for you
man resolv.conf to see more info on how to get multiple nameservers in
resolv.conf
here i have just installed bind and set nameserver to 127.0.0.1 :-)
Re: how to disable network tests?
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2011-03-18 at 02:38 +0400, Hamad Ali wrote:
> > I take it you no longer want to disable network test? ;)
>
> yeah, thank you for all the help; also thanks to other folks that
> told me how to disable network tests (it helped me see the need of
> network tests).
Great. To see a proper solution for the *underlying* problem (no matter
the Subject), as well as hopefully lessons learned.
Back to a pet peeve of mine. The imminent question of a user might not
lead to the best solution. Frankly, quite often, it does not. It is
important for the actively helping community, as well as best results
for the OP, to understand the real issue. For that, we do ask questions,
and propose things that might on a first glance, for the user, be
unrelated. Though it frequently turns out to be spot on.
In a community, like this, people try to help on their own, often spare
time, voluntarily. Pissing those off will just lead to them turning to
the TV instead of giving it another shot with you again next time. ;)
guenther
Oh, and I really don't wanna fuel the fire any further, but -- most [1]
of the gang around seriously prefers old-school plain-text mail (HTML
disabled) and reply-to-list only, rather than Cc the sender. We do read
the list. But yeah, well, those are minor, comparably...
[1] Obviously, not everyone. And particularly with the second note,
there are even folks who *cough* want the Cc. Subtle things that
sometimes only might surface after a while, if one actually even
recognizes guys and their preference.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
RE: how to disable network tests?
Posted by Hamad Ali <cr...@hotmail.com>.
> Subject: Re: how to disable network tests?
> From: guenther@rudersport.de
> To: users@spamassassin.apache.org
> Date: Thu, 17 Mar 2011 23:04:59 +0100
> Did you clear your local (forwarding) DNS's cache between these runs?
> Hmm, or maybe both are essentially suffering from the same major
> non-services.
Yeah, /etc/init.d/named restart.
> I take it you no longer want to disable network test? ;)
yeah, thank you for all the help; also thanks to other folks that told me how to disable network tests (it helped me see the need of network tests).
Re: how to disable network tests?
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2011-03-18 at 01:43 +0400, Hamad Ali wrote:
> > [...] Does your caching DNS act as a forwarder?
> >
> > Don't do that. Run a local, caching, non-forwarding DNS server. And make
> > sure this server (most likely 127.0.0.1) is the first nameserver entry
> > in your /etc/resolv.conf.
>
> Yeah, I was forwarding to my ISP's and Google's DNS servers, but never
> tried turning off forwarding. These are the worst results I got from
> multiple runs against an email that used to take a lot of time:
>
> # forwarding t o my ISP's DNS server
> time cat 00005.mail | spamc -c
> real 0m10.047s
> # forwarding to google's DNS servers
> time cat 0000 5.mail | spamc -c
> real 0m12.062s
Did you clear your local (forwarding) DNS's cache between these runs?
Hmm, or maybe both are essentially suffering from the same major
non-services.
> # no forwarders
> time cat 00005.mail | spamc -c
> real 0m5.717s
Seems reasonable. (And keep in mind, unless you're severely RAM or CPU
constrained, multiple messages can be scanned simultaneously. An
additional second or two waiting for DNS queries is just idle waiting.)
> I can understand why Google's DNS takes up to 12 seconds (located
> geographically far away), but surprised to see that my ISP's server is
> worst than not using forwarders (I was expecting to take advantage of
> a nearby cache). Probably I am rate-limited by my ISP's DNS.
ISP's DNS servers tend to be blocked by some of the major DNSBLs, due to
excessive usage. It is always advisable to run a local resolver (aka
non-caching) for best results.
I take it you no longer want to disable network test? ;)
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
RE: how to disable network tests?
Posted by Hamad Ali <cr...@hotmail.com>.
> Subject: RE: how to disable network tests?
> From: guenther@rudersport.de
> To: users@spamassassin.apache.org
> Date: Thu, 17 Mar 2011 21:30:16 +0100
> The new timings strongly suggest there indeed are some network or DNS
> issues. Installing a local caching nameserver already helped.
>
> However, you just mentioned "using other DNS servers than your ISP's
> one" in that very context. Does your caching DNS act as a forwarder?
>
> Don't do that. Run a local, caching, non-forwarding DNS server. And make
> sure this server (most likely 127.0.0.1) is the first nameserver entry
> in your /etc/resolv.conf.
Yeah, I was forwarding to my ISP's and Google's DNS servers, but never tried turning off forwarding. These are the worst results I got from multiple runs against an email that used to take a lot of time:
# forwarding to my ISP's DNS servertime cat 00005.mail | spamc -c 13.7/5.0
real 0m10.047suser 0m0.020ssys 0m0.000s
# forwarding to google's DNS serverstime cat 00005.mail | spamc -c 13.7/5.0
real 0m12.062suser 0m0.020ssys 0m0.000s
# no forwarderstime cat 00005.mail | spamc -c 13.7/5.0
real 0m5.717suser 0m0.020ssys 0m0.000s
I can understand why Google's DNS takes up to 12 seconds (located geographically far away), but surprised to see that my ISP's server is worst than not using forwarders (I was expecting to take advantage of a nearby cache). Probably I am rate-limited by my ISP's DNS.
Thank you for all your support!
-- H
RE: how to disable network tests?
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
Oh, and FWIW...
> On Thu, 2011-03-17 at 22:17 +0400, Hamad Ali wrote:
> > Regarding my problem, I am still investigating the cause of the
> > network delay. I have installed a DNS cac he server, and speed did
> > increase for "cached" entries. [...]
This is precisely what you should have posted in the first place.
Showing you did listen to the advice, tried something, and made progress
narrowing down the issue -- even if there's still some way to go.
It's not necessarily about a "thank you" we'd expect -- but results of
the various hints that have been posted, trying to help you.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: how to disable network tests?
Posted by Mark Martinec <Ma...@ijs.si>.
> > when local dns server is working there must only be one single nameserver
> > in resolv.conf and options rotate nust not be enabled
>
On Friday March 18 2011 04:04:23 Karsten Bräckelmann wrote:
> In the SA case, due to some issues with the underlying DNS Perl module,
> IIRC the "first nameserver is all that ever will be used" note is the
> appropriate disclaimer -- though I got to admit, details escape me.
The "first nameserver is all that ever will be used" is true for 3.3.1
and older.
This restriction has been lifted in trunk (3.4), an automatic failover
between configured nameservers now works, and DNS servers can now be
specified in a SpamAssassin config file (directives dns_server and
clear_dns_servers), including IPv6 servers, if necessary with server
port numbers. Restrictions on client port ranges can be specified
(directive dns_local_ports_permit) to deal with firewalls.
(discussed in Bug 6362 and thereabout).
Mark
Re: how to disable network tests?
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2011-03-18 at 01:49 +0100, Benny Pedersen wrote:
> On Thu, 17 Mar 2011 21:30:16 +0100, Karsten Bräckelmann wrote:
>
> > Don't do that. Run a local, caching, non-forwarding DNS server. And make
> > sure this server (most likely 127.0.0.1) is the first nameserver entry
> > in your /etc/resolv.conf.
>
> when local dns server is working there must only be one single nameserver
> in resolv.conf and options rotate nust not be enabled
In the SA case, due to some issues with the underlying DNS Perl module,
IIRC the "first nameserver is all that ever will be used" note is the
appropriate disclaimer -- though I got to admit, details escape me.
> one nameserver:
>
> nameserver 127.0.0.1
> options without rotate
>
> two or more nameservers:
>
> nameserver 10.0.0.1
> nameserver 10.0.0.2
> options rotate
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
RE: how to disable network tests?
Posted by Benny Pedersen <me...@junc.org>.
On Thu, 17 Mar 2011 21:30:16 +0100, Karsten Bräckelmann
<gu...@rudersport.de> wrote:
> Don't do that. Run a local, caching, non-forwarding DNS server. And make
> sure this server (most likely 127.0.0.1) is the first nameserver entry
> in your /etc/resolv.conf.
when local dns server is working there must only be one single nameserver
in resolv.conf and options rotate nust not be enabled
one nameserver:
nameserver 127.0.0.1
options without rotate
two or more nameservers:
nameserver 10.0.0.1
nameserver 10.0.0.2
options rotate
RE: how to disable network tests?
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Thu, 2011-03-17 at 22:17 +0400, Hamad Ali wrote:
> Thank you all for your extensive support. I have to say that this
> thread told me that SA's list is one of the most lists out there.
[...]
> Regarding my problem, I am still investigating the cause of the
> network delay. I have installed a DNS cac he server, and speed did
> increase for "cached" entries. Sometimes, I get ~1.4 seconds (time(1))
> for email scanning with network enabled, and 0.1 seconds when network
> tests are disabled. I tried other DNS servers beside my ISP's, and
> they all were similarly slow for non-cached entries. I am suspecting
> that my SP is doing something weird, it could be that they are doing
> some form of transparent DNS proxying to filter unwanted websites.
Now this is better. :) You see, all these comments to your original
post do have a reason -- we can help better, if they get answered,
because they can highlight the actual problem. Ignoring those offering
help, on the other hand, might get some folks upset.
The new timings strongly suggest there indeed are some network or DNS
issues. Installing a local caching nameserver already helped.
However, you just mentioned "using other DNS servers than your ISP's
one" in that very context. Does your caching DNS act as a forwarder?
Don't do that. Run a local, caching, non-forwarding DNS server. And make
sure this server (most likely 127.0.0.1) is the first nameserver entry
in your /etc/resolv.conf.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
RE: how to disable network tests?
Posted by Hamad Ali <cr...@hotmail.com>.
Thank you all for your extensive support. I have to say that this thread told me that SA's list is one of the most lists out there.
Why didn't I reply back? I initially wanted to reply to thank you for the great friendly free help (at least). However, then I thought that sending a "thank you" mail is not productive, and it might waste the time of all the participants of the list --- that's all! So I changed my mind, changed subject/content and sent my other question for sake of optimizing time usage (specially that my 1st problem is only partially solved by now).
Regarding my problem, I am still investigating the cause of the network delay. I have installed a DNS cache server, and speed did increase for "cached" entries. Sometimes, I get ~1.4 seconds (time(1)) for email scanning with network enabled, and 0.1 seconds when network tests are disabled. I tried other DNS servers beside my ISP's, and they all were similarly slow for non-cached entries. I am suspecting that my SP is doing something weird, it could be that they are doing some form of transparent DNS proxying to filter unwanted websites.
-- H
> Date: Sun, 13 Mar 2011 01:25:13 +0000> From: rwmaillists@googlemail.com
> To: users@spamassassin.apache.org
> Subject: Re: how to disable network tests?
>
> On Sat, 12 Mar 2011 15:00:57 -0600 (CST)
> Dave Funk <db...@engineering.uiowa.edu> wrote:
>
> > On Sat, 12 Mar 2011, RW wrote:
> >
> > > On Fri, 11 Mar 2011 13:39:58 -1000
> > > "Warren Togami Jr." <wt...@gmail.com> wrote:
> > >
> > >> If it is taking 10 seconds per message then you likely have
> > >> some kind of serious misconfiguration.
> > >
> > >
> > > It depends on the hardware, there's plenty of SOHO hardware where
> > > 10s would be considered snappy.
> > >
> >
> > True, if running on under-powered or memory-limited hardware then
> > the 10 second time would be explainable. But in that case you'd see
> > high load-aves.
>
> Not noticing a transient high load-average is consistent with a SOHO
> server, desktop, or mail client plugin. My desktop PC takes 15+ seconds
> and spends most of its time in REs but I never notice a slowdown, even
> though it's a single core.
>
> > If the box is mostly waiting for net-test results
> > that's a config issue (or serious network problem ;).
> >
> > The OP did ask about disabling all network tests which would imply
> > that he had some reason to suspect the latter.
>
>
> You pretty much either know it's a network problem, or you're guessing.
> And if you know it's a network problem, it's a short step to knowing
> why, so my guess is that it's supposition.
>
> I think CPU limiting is a distinct possibility.
>
>
Re: Performance on Spear Phishing?
Posted by "Warren Togami Jr." <wt...@gmail.com>.
On 3/16/2011 4:08 PM, Hamad Ali wrote:
> Hi folks -- wondering if anyone has monitored SA's performance against
> phishing mails. SA is able to detect 86% of phishing emails my clients
> get, with 0.5% false positives on all the ham. It seems non-phish-SPAM
> is easier to be detected than phish (~99% for non-phish spam). Probably
> I need to participate on nightly checks to improve phish and lower false
> positives.
>
> But all the above stuff is about bulk-phish, excluding spear phish. I
> haven't received any spear phishing complain from my clients, and yet
> none of the detected phish mails are spear phish -- which is alarming as
> it's too good to be true that no one did spear phishing yet (specially
> that it works far better than bulk-phish)!
>
> What's the scenario in your mail systems folks? Do you detect spear
> phishing mail by SA? Users report it?
>
> -- H
>
>
Are you using spamassassin-3.3.1?
http://www.spamtips.org/p/ultimate-setup-guide.html
Have you tweaked it with the best tested add-ons? Please read this page.
In particular the fuzzy hash based plugins like pyzor, Razor and DCC
sometimes is effective against phishing.
Warren
Re: Performance on Spear Phishing?
Posted by Nigel Frankcom <ni...@blue-canoe.com>.
On Fri, 18 Mar 2011 04:22:40 +0100, Karsten Bräckelmann
<gu...@rudersport.de> wrote:
>On Thu, 2011-03-17 at 12:58 +0000, Nigel Frankcom wrote:
>> Unrelated but reminded me I hadn't posted a thanks to all those that
>> responded about the sa-update rules. That's partly because I'm
>> awaiting permission from clients to add their mails to the corpus.
>
>Unrelated indeed. ;) That short rant of mine was not meant as a broad
>reminder to send your 'thank you's after each post, less so to collect
>them now -- but really triggered by that one particular instance.
>
>There are a bunch of circumstances (some slightly buried down the end)
>outlined in my previous post, which, each on their own, if avoided, are
>likely to not have triggered my reaction in the first place. In other
>words, just try to engage in the community, and don't forget basic
>(old-school) net-iquette, and we all should get along just fine. :)
>
>> So, thanks all. Apologies for forgetting my manners.
>>
>> Have no clue about Spear Phishing other than it's best to be the one
>> with the spear. :-)
>
>Or the hammer.
Hi Karsten,
Having been using this list for more years than I care to think about
I ought to know my manners better. It was a timely reminder, it's easy
to take the help one gets here for granted. I don't tend to post so
much nowadays with workloads etc, but it's the only list I stay
subscribed to. I do on occasion sit with a beer on a boring evening
and amble through the posts, and, occasionally, I note things with my
setup that seem a bit off.
Without wishing to tempt fate, my setup works well for me and works
well. Often as not because of advice given in the past by list
members; anyway, manners cost nothing and they do have a value for the
recipients.
All the best
Nigel
Re: Performance on Spear Phishing?
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Thu, 2011-03-17 at 12:58 +0000, Nigel Frankcom wrote:
> Unrelated but reminded me I hadn't posted a thanks to all those that
> responded about the sa-update rules. That's partly because I'm
> awaiting permission from clients to add their mails to the corpus.
Unrelated indeed. ;) That short rant of mine was not meant as a broad
reminder to send your 'thank you's after each post, less so to collect
them now -- but really triggered by that one particular instance.
There are a bunch of circumstances (some slightly buried down the end)
outlined in my previous post, which, each on their own, if avoided, are
likely to not have triggered my reaction in the first place. In other
words, just try to engage in the community, and don't forget basic
(old-school) net-iquette, and we all should get along just fine. :)
> So, thanks all. Apologies for forgetting my manners.
>
> Have no clue about Spear Phishing other than it's best to be the one
> with the spear. :-)
Or the hammer.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Performance on Spear Phishing?
Posted by Nigel Frankcom <ni...@blue-canoe.com>.
Unrelated but reminded me I hadn't posted a thanks to all those that
responded about the sa-update rules. That's partly because I'm
awaiting permission from clients to add their mails to the corpus.
So, thanks all. Apologies for forgetting my manners.
Have no clue about Spear Phishing other than it's best to be the one
with the spear. :-)
On Thu, 17 Mar 2011 04:38:29 +0100, Karsten Bräckelmann
<gu...@rudersport.de> wrote:
>So this actually is a reply to the last post to your previous thread
>"how to disable network tests". Merely changing the subject and pruning
>the quote from the body -- surprise -- does NOT make it a new thread. On
>the up-side, it appears you at least did read (I mean "keep" here) the
>thread. Encouraging.
>
>There has been a lot of help, advice, and questions concerning your
>previous topic, however. The down-side. You did not care to even get
>back to a single one of them. Very discouraging.
>
>Do you really expect anyone to care and try to help a single-shot
>question you vent on the list again?
>
>I for one, bloody don't.
>
>
>On Thu, 2011-03-17 at 06:08 +0400, Hamad Ali wrote:
>> Hi folks -- wondering if anyone has monitored SA's performance against
>> phishing mails. SA is able to detect 86% of phishing emails my clients
>
>So you got paying clients. But won't communicate with the community.
>
>> get, with 0.5% false positives on all the ham. It seems non-phish-SPAM
>> is easier to be detected than phish (~99% for non-phish spam). Probably
>> I need to participate on nightly checks to improve phish and lower
>> false positives.
>
>Participating in the mass-checks!? Without any communication (hint, two
>ways) at all? I don't see that happening.
Re: Performance on Spear Phishing?
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
So this actually is a reply to the last post to your previous thread
"how to disable network tests". Merely changing the subject and pruning
the quote from the body -- surprise -- does NOT make it a new thread. On
the up-side, it appears you at least did read (I mean "keep" here) the
thread. Encouraging.
There has been a lot of help, advice, and questions concerning your
previous topic, however. The down-side. You did not care to even get
back to a single one of them. Very discouraging.
Do you really expect anyone to care and try to help a single-shot
question you vent on the list again?
I for one, bloody don't.
On Thu, 2011-03-17 at 06:08 +0400, Hamad Ali wrote:
> Hi folks -- wondering if anyone has monitored SA's performance against
> phishing mails. SA is able to detect 86% of phishing emails my clients
So you got paying clients. But won't communicate with the community.
> get, with 0.5% false positives on all the ham. It seems non-phish-SPAM
> is easier to be detected than phish (~99% for non-phish spam). Probably
> I need to participate on nightly checks to improve phish and lower
> false positives.
Participating in the mass-checks!? Without any communication (hint, two
ways) at all? I don't see that happening.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Performance on Spear Phishing?
Posted by John Hardin <jh...@impsec.org>.
On Wed, 16 Mar 2011, Warren Togami Jr. wrote:
> On 3/16/2011 5:45 PM, Karsten Bräckelmann wrote:
>> On Wed, 2011-03-16 at 20:30 -0700, John Hardin wrote:
>> > On Thu, 17 Mar 2011, Hamad Ali wrote:
>>
>> > > Probably I need to participate on nightly checks to improve phish and
>> > > lower false positives.
>> >
>> > More masscheck participants are always welcome!
>>
>> No.
>>
>> There is this thing called trust. Credibility. And track-record. Which
>> pretty much is the opposite of a freemail address, venting two questions
>> on this list -- without ever getting back even to specific requests for
>> better data, offer for precise help, or a dialog.
>
> Karsten, thanks for pointing out that this is the same guy. I had missed
> that.
As did I.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Markley's Law (variant of Godwin's Law): As an online discussion
of gun owners' rights grows longer, the probability of an ad hominem
attack involving penis size approaches 1.
-----------------------------------------------------------------------
12 days until the M1911 is 100 years old - and still going strong!
Re: Performance on Spear Phishing?
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Wed, 2011-03-16 at 17:50 -1000, Warren Togami Jr. wrote:
> Karsten, thanks for pointing out that this is the same guy. I had
> missed that.
Heh, you're welcome -- though that would be referring to my other reply
to this (sub-) thread. ;)
Sometimes it helps to identify patterns. Sometimes it helps to use
threaded list view, especially with mailing lists. And sometimes it
helps to get offended, pissed-off, or just annoyed by certain non-
communicating free-loader behavior. The latter develops over time.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Performance on Spear Phishing?
Posted by Michael Scheidell <mi...@secnap.com>.
On 3/16/11 11:50 PM, Warren Togami Jr. wrote:
>>
>
> Karsten, thanks for pointing out that this is the same guy. I had
> missed that.
>
> Warren
Ditto. I was about to tell him how to stop spear phishing.
thanks.
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
ISN: 1259*1300
>*| *SECNAP Network Security Corporation
* Best Intrusion Prevention Product, Networks Product Guide
* Certified SNORT Integrator
* Hot Company Award, World Executive Alliance
* Best in Email Security, 2010 Network Products Guide
* King of Spam Filters, SC Magazine
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________
Re: Performance on Spear Phishing?
Posted by "Warren Togami Jr." <wt...@gmail.com>.
On 3/16/2011 5:45 PM, Karsten Bräckelmann wrote:
> On Wed, 2011-03-16 at 20:30 -0700, John Hardin wrote:
>> On Thu, 17 Mar 2011, Hamad Ali wrote:
>
>>> Probably I need to participate on nightly checks to improve phish and
>>> lower false positives.
>>
>> More masscheck participants are always welcome!
>
> No.
>
> There is this thing called trust. Credibility. And track-record. Which
> pretty much is the opposite of a freemail address, venting two questions
> on this list -- without ever getting back even to specific requests for
> better data, offer for precise help, or a dialog.
>
>
Karsten, thanks for pointing out that this is the same guy. I had
missed that.
Warren
Re: Performance on Spear Phishing?
Posted by John Hardin <jh...@impsec.org>.
On Thu, 17 Mar 2011, Karsten Br�ckelmann wrote:
> On Wed, 2011-03-16 at 20:30 -0700, John Hardin wrote:
>> On Thu, 17 Mar 2011, Hamad Ali wrote:
>
>>> Probably I need to participate on nightly checks to improve phish and
>>> lower false positives.
>>
>> More masscheck participants are always welcome!
>
> No.
>
> There is this thing called trust. Credibility. And track-record. Which
> pretty much is the opposite of a freemail address, venting two questions
> on this list -- without ever getting back even to specific requests for
> better data, offer for precise help, or a dialog.
Fair enough.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Markley's Law (variant of Godwin's Law): As an online discussion
of gun owners' rights grows longer, the probability of an ad hominem
attack involving penis size approaches 1.
-----------------------------------------------------------------------
12 days until the M1911 is 100 years old - and still going strong!
Re: Performance on Spear Phishing?
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Wed, 2011-03-16 at 20:30 -0700, John Hardin wrote:
> On Thu, 17 Mar 2011, Hamad Ali wrote:
> > Probably I need to participate on nightly checks to improve phish and
> > lower false positives.
>
> More masscheck participants are always welcome!
No.
There is this thing called trust. Credibility. And track-record. Which
pretty much is the opposite of a freemail address, venting two questions
on this list -- without ever getting back even to specific requests for
better data, offer for precise help, or a dialog.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Performance on Spear Phishing?
Posted by John Hardin <jh...@impsec.org>.
On Thu, 17 Mar 2011, Hamad Ali wrote:
> Hi folks -- wondering if anyone has monitored SA's performance against
> phishing mails. SA is able to detect 86% of phishing emails my clients
> get, with 0.5% false positives on all the ham. It seems non-phish-SPAM
> is easier to be detected than phish (~99% for non-phish spam).
I think phishing is going to be my next project.
> Probably I need to participate on nightly checks to improve phish and
> lower false positives.
More masscheck participants are always welcome!
> But all the above stuff is about bulk-phish, excluding spear phish. I
> haven't received any spear phishing complain from my clients, and yet
> none of the detected phish mails are spear phish -- which is alarming as
> it's too good to be true that no one did spear phishing yet (specially
> that it works far better than bulk-phish)!
Spear-phishing is probably going to be rather difficult to detect, I'm not
sure even a well-trained Bayes would help.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Gun Control is nothing more than an attempt to return to feudalism,
where the peasants are helpless and must humbly petition their lord
and master to protect them from bandits and thieves (when they can
get around to it), and where the lords and masters can abuse the
peasants whenever they like without fear of effective resistance.
-----------------------------------------------------------------------
13 days until the M1911 is 100 years old - and still going strong!
Performance on Spear Phishing?
Posted by Hamad Ali <cr...@hotmail.com>.
Hi folks -- wondering if anyone has monitored SA's performance against phishing mails. SA is able to detect 86% of phishing emails my clients get, with 0.5% false positives on all the ham. It seems non-phish-SPAM is easier to be detected than phish (~99% for non-phish spam). Probably I need to participate on nightly checks to improve phish and lower false positives.
But all the above stuff is about bulk-phish, excluding spear phish. I haven't received any spear phishing complain from my clients, and yet none of the detected phish mails are spear phish -- which is alarming as it's too good to be true that no one did spear phishing yet (specially that it works far better than bulk-phish)!
What's the scenario in your mail systems folks? Do you detect spear phishing mail by SA? Users report it?
-- H
Re: how to disable network tests?
Posted by jdow <jd...@earthlink.net>.
On 2011/03/12 17:25, RW wrote:
> On Sat, 12 Mar 2011 15:00:57 -0600 (CST)
> Dave Funk<db...@engineering.uiowa.edu> wrote:
>
>> On Sat, 12 Mar 2011, RW wrote:
>>
>>> On Fri, 11 Mar 2011 13:39:58 -1000
>>> "Warren Togami Jr."<wt...@gmail.com> wrote:
>>>
>>>> If it is taking 10 seconds per message then you likely have
>>>> some kind of serious misconfiguration.
>>>
>>>
>>> It depends on the hardware, there's plenty of SOHO hardware where
>>> 10s would be considered snappy.
>>>
>>
>> True, if running on under-powered or memory-limited hardware then
>> the 10 second time would be explainable. But in that case you'd see
>> high load-aves.
>
> Not noticing a transient high load-average is consistent with a SOHO
> server, desktop, or mail client plugin. My desktop PC takes 15+ seconds
> and spends most of its time in REs but I never notice a slowdown, even
> though it's a single core.
>
>> If the box is mostly waiting for net-test results
>> that's a config issue (or serious network problem ;).
>>
>> The OP did ask about disabling all network tests which would imply
>> that he had some reason to suspect the latter.
>
>
> You pretty much either know it's a network problem, or you're guessing.
> And if you know it's a network problem, it's a short step to knowing
> why, so my guess is that it's supposition.
>
> I think CPU limiting is a distinct possibility.
I haven't exceeded ten second SpamAssassin passes since I was using
pure spamassassin and not spamc to scan mail on a very slow Pentium
with very limited memory. And I tend to run just a whole LOT of rules.
With 45 sets of rules plus network tests SpamAssassin 3.2.5 takes 7+
seconds and spamc takes 3.5 seconds "typically". That's on an old
Athlon clocked at 1.8 GHz with 1G of memory that is on its last legs.
(It's at least a decade old.)
{o.o}
Re: how to disable network tests?
Posted by RW <rw...@googlemail.com>.
On Sat, 12 Mar 2011 15:00:57 -0600 (CST)
Dave Funk <db...@engineering.uiowa.edu> wrote:
> On Sat, 12 Mar 2011, RW wrote:
>
> > On Fri, 11 Mar 2011 13:39:58 -1000
> > "Warren Togami Jr." <wt...@gmail.com> wrote:
> >
> >> If it is taking 10 seconds per message then you likely have
> >> some kind of serious misconfiguration.
> >
> >
> > It depends on the hardware, there's plenty of SOHO hardware where
> > 10s would be considered snappy.
> >
>
> True, if running on under-powered or memory-limited hardware then
> the 10 second time would be explainable. But in that case you'd see
> high load-aves.
Not noticing a transient high load-average is consistent with a SOHO
server, desktop, or mail client plugin. My desktop PC takes 15+ seconds
and spends most of its time in REs but I never notice a slowdown, even
though it's a single core.
> If the box is mostly waiting for net-test results
> that's a config issue (or serious network problem ;).
>
> The OP did ask about disabling all network tests which would imply
> that he had some reason to suspect the latter.
You pretty much either know it's a network problem, or you're guessing.
And if you know it's a network problem, it's a short step to knowing
why, so my guess is that it's supposition.
I think CPU limiting is a distinct possibility.
Re: how to disable network tests?
Posted by Dave Funk <db...@engineering.uiowa.edu>.
On Sat, 12 Mar 2011, RW wrote:
> On Fri, 11 Mar 2011 13:39:58 -1000
> "Warren Togami Jr." <wt...@gmail.com> wrote:
>
>> If it is taking 10 seconds per message then you likely have
>> some kind of serious misconfiguration.
>
>
> It depends on the hardware, there's plenty of SOHO hardware where 10s
> would be considered snappy.
>
True, if running on under-powered or memory-limited hardware then
the 10 second time would be explainable. But in that case you'd see
high load-aves. If the box is mostly waiting for net-test results
that's a config issue (or serious network problem ;).
The OP did ask about disabling all network tests which would imply
that he had some reason to suspect the latter.
--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{
Re: how to disable network tests?
Posted by RW <rw...@googlemail.com>.
On Fri, 11 Mar 2011 13:39:58 -1000
"Warren Togami Jr." <wt...@gmail.com> wrote:
> If it is taking 10 seconds per message then you likely have
> some kind of serious misconfiguration.
It depends on the hardware, there's plenty of SOHO hardware where 10s
would be considered snappy.
Re: how to disable network tests?
Posted by "Warren Togami Jr." <wt...@gmail.com>.
On 3/11/2011 10:05 AM, Hamad Ali wrote:
> hi folks --- everything seems working like chicken. I'm loving SA so far.
>
> However, I would like to disable all network tests (each mail takes ~10
> seconds!). Except that I dunno how to do it the neat way.
>
> Will the tests be disabled if their score is 0? I know that would lead
> into disabling the effect of a rule on the decision making of SA (i.e.
> Spam/Ham marking), but would SA exclude them from running too?
>
> I need to disable all BLs, DNS queries, and anything that uses the
> internet. Kindly advise.
>
> Thank you guys -- May OOP Raise and Shine!
> H
Please consider that spamassassin is CRIPPLED without the network tests.
If it is taking 10 seconds per message then you likely have some kind
of serious misconfiguration. The first likely culprit is your DNS
server is not good. Several times in past years I've had to stop using
my ISP's (or data center's!) official DNS servers because they were
simply not capable of handling the load of spamassassin. In such cases
I run pdns-recursor on each Spamassassin server directly, and set
/etc/resolv.conf to use 127.0.0.1 as the DNS resolver.
After you have switched to a known good DNS server, do the following to
diagnose the network tests.
1) Save a single spam message as a flat file, with headers and body
intact. If your folders are Maildir format then a single file in your
directory tree is suitable for this purpose.
2) cat FILE | spamassassin -D
3) Copy the entire output and paste into a text editor.
4) Look at the lines near the bottom for "async: timing:" Those are
followed by a number of seconds that an individual DNS request took to
respond. All of these numbers are typically between 0 and 3 seconds on
my server. If you have much larger numbers or some queries are timing
out entirely, then you may have further issues with your DNS server, or
you may have been blocked from queries because you have exceeded free
usage limits.
http://www.spamtips.org/2011/01/usage-limits-of-spamassassin-network.html
Please see my article here about the free usage limits of the various
spamassassin network tests.
http://www.spamtips.org/p/ultimate-setup-guide.html
Please read this page for all known safe and effective configuration
tweaks to spamassassin.
Warren Togami
warren@togami.com
Re: how to disable network tests?
Posted by jdow <jd...@earthlink.net>.
On 2011/03/11 13:02, David F. Skoll wrote:
> On Fri, 11 Mar 2011 12:51:44 -0800 (PST)
> John Hardin<jh...@impsec.org> wrote:
>
>> ...your email is so time-critical that you can't wait an extra ten
>> seconds for it to be delivered?
>
> On a busy server, a ten-second latency in scanning mail could kill you...
>
> As another poster said, 10s for network tests seems excessive. There's
> probably something wrong.
More specifically, he is probably trying to use a moribund DNS test
site. He should review what he is using, testing each of the sites
to make sure they are all valid.
{^_^}
Re: how to disable network tests?
Posted by "David F. Skoll" <df...@roaringpenguin.com>.
On Fri, 11 Mar 2011 12:51:44 -0800 (PST)
John Hardin <jh...@impsec.org> wrote:
> ...your email is so time-critical that you can't wait an extra ten
> seconds for it to be delivered?
On a busy server, a ten-second latency in scanning mail could kill you...
As another poster said, 10s for network tests seems excessive. There's
probably something wrong.
Regards,
David.
Re: how to disable network tests?
Posted by John Hardin <jh...@impsec.org>.
On Sat, 12 Mar 2011, Hamad Ali wrote:
> hi folks --- everything seems working like chicken. I'm loving SA so far.
Glad to hear it!
> However, I would like to disable all network tests (each mail takes ~10 seconds!).
...your email is so time-critical that you can't wait an extra ten seconds
for it to be delivered?
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Failure to plan ahead on someone else's part does not constitute
an emergency on my part. -- David W. Barts in a.s.r
-----------------------------------------------------------------------
2 days until Daylight Saving Time begins in U.S. - Spring Forward
Re: how to disable network tests?
Posted by Andrzej Adam Filip <an...@gmail.com>.
Hamad Ali <cr...@hotmail.com> wrote:
> hi folks --- everything seems working like chicken. I'm loving SA so
> far.
>
> However, I would like to disable all network tests (each mail takes ~10
> seconds!). Except that I dunno how to do it the neat way.
>
> Will the tests be disabled if their score is 0? I know that would lead
> into disabling the effect of a rule on the decision making of SA (i.e.
> Spam/Ham marking), but would SA exclude them from running too?
>
> I need to disable all BLs, DNS queries, and anything that uses the
> internet. Kindly advise.
>
> Thank you guys -- May OOP Raise and Shine!
How do you run spamassassin?
* for all users on your server/for your account only
* at MTA level (milter) or via procmail
You can use -L command line option to turn off all non local tests.
Be warn: it will reduce SA efficiency in spam detection
--
[pl>en: Andrew] Andrzej Adam Filip : anfi@onet.eu
We are what we pretend to be.
-- Kurt Vonnegut, Jr.
Re: how to disable network tests?
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Sat, 2011-03-12 at 00:05 +0400, Hamad Ali wrote:
> However, I would like to disable all network tests (each mail takes
> ~10 seconds!). Except that I dunno how to do it the neat way.
That seems a little bit excessive. Are you sure this is due to DNS
queries?
If so, it would appear you are having a problem with your DNS server.
Not performing network checks would merely be a band-aid, fixing the DNS
issue would be much more effective. (Note that the first 'nameserver'
entry from /etc/resolv.conf is being used exclusively.)
> Will the tests be disabled if their score is 0? I know that would lead
> into disabling the effect of a rule on the decision making of SA (i.e.
> Spam/Ham marking), but would SA exclude them from running too?
Merely setting the scores to 0 is not sufficient, some sub-rules still
would be run.
Disabling network tests will result in SA using a different score-set,
and thus scores for all rules still being run.
> I need to disable all BLs, DNS queries, and anything that uses the
> internet. Kindly advise.
skip_rbl_checks 1
skip_uribl_checks 1 # since SA 3.3.x
Especially with SA 3.2.x, the following might help, too. Otherwise,
URIBL checks might still be performed, IIRC.
dns_available no
Other than the above config (to be placed in site-wide local.cf),
there's also the spamd -L option for local checks only.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}