[Bug 3881] dnsbl lookups slow due to "sleep 1" in code

[bu...@bugzilla.spamassassin.org]

http://bugzilla.spamassassin.org/show_bug.cgi?id=3881


quinlan@pathname.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dev@spamassassin.apache.org
         AssignedTo|dev@spamassassin.apache.org |quinlan@pathname.com




------- Additional Comments From quinlan@pathname.com  2005-02-25 12:20 -------
To summarize my (remaining) earlier comments made back in October on
users@spamassassin.apache.org:

1. My main concern is the increase in CPU time required (since multiple
   spamd processes will be running), especially on boxes that don't have
   local DNSBL mirrors.

   My testing of the patch using mass-check -j 8 --net has backed up this
   concern.  Both CPU and wall time increase with this patch.  :-/

2. The timeout scales dynamically based on how many blacklists have already
   replied (see the rbl_timeout documentation).  If you have enough of them
   locally mirrored, then you'll almost never see results from external
   blacklists unless the non-network tests required more time (because the
   timeout may scale so fast that the non-local tests never get a chance to
   return like they would with sleep 1).

Right now, I'm -0.4 on applying this patch.  I think the additional slowdown
primarily comes from entering the processing loop more times per blacklist
plus some overhead of more frequent context switches between SpamAssassin
child processes.  It might be possible to alleviate the downsides by using
sleep (either at beginning or end of checks, not sure) instead of the harvest
period -- some way to chunk things up better.  (You'd have to look at the
average profile/distribution of DNSBL response times.)

A simple sub-second sleep might also be a way to get there, use subsecond
sleep (0.5 seconds?) instead of the current sleep 1.

SA devs: any thoughts?



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
You are on the CC list for the bug, or are watching someone who is.