You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by "Benning, Markus" <ic...@markusbenning.de> on 2015/06/09 17:57:11 UTC
[Announce] SA-Plugins: RedisAWL, RuleTimingRedis
Hello,
i want to announce the release of the SpamAssassin Plugins:
RedisAWL - redis support for spamassassin AWL/TxRep
RuleTimingRedis - collect SA rule timings in redis
Both can be downloaded from CPAN or GitHub:
https://metacpan.org/author/BENNING
https://github.com/benningm
Timings gathered with the RuleTimingRedis plugin can be used in collectd
with the Collectd-Plugins-RedisClient module also available from CPAN.
Markus
--
Markus Benning, https://markusbenning.de/
Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis
Posted by RW <rw...@googlemail.com>.
On Tue, 09 Jun 2015 21:35:18 +0200
Axb wrote:
>
> suggest you provide a dedicated redisawl.pre including loadplugin
> command
>
> init.pre can be overwritten by an upgrade
That's not supposed to happen.
Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis
Posted by Axb <ax...@gmail.com>.
On 09.06.2015 17:57, Benning, Markus wrote:
> Hello,
>
> i want to announce the release of the SpamAssassin Plugins:
>
> RedisAWL - redis support for spamassassin AWL/TxRep
> RuleTimingRedis - collect SA rule timings in redis
>
> Both can be downloaded from CPAN or GitHub:
>
> https://metacpan.org/author/BENNING
>
> https://github.com/benningm
>
> Timings gathered with the RuleTimingRedis plugin can be used in collectd
> with the Collectd-Plugins-RedisClient module also available from CPAN.
>
> Markus
>
interesting
suggest you provide a dedicated redisawl.pre including loadplugin command
init.pre can be overwritten by an upgrade
What kind of traffic have you tested this with?
User count?
Msgs/day/week/month?
Imo, that kind of thing is important with stufff using Redis as you can
easily max out a machine's memory capacity.
Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis
Posted by Patrick Ben Koetter <p...@sys4.de>.
Markus,
* Benning, Markus <ic...@markusbenning.de>:
> Hi Patrik,
>
> i just pushed Version 1.002 to github and CPAN:
>
> --
> The following new features have been added:
>
> - New option: timing_redis_password allows to specifiy a redis
> password
>
> - New option: timing_redis_exclude_re excludes rules from timing
> statistics. By default set to '^__' which will exclude all sub-rules
>
> - New option: timing_redis_database allows to select a non-default
> database in redis. (redis SWITCH call)
>
> - New option: timing_redis_bulk_update will queue timing updates
> before sending them to redis and execute them in a bulk via a
> single call to a server-side script. By default this option is set
> to 50 entries. Set to 0 do disable. Requires redis >= 2.6.0 and a
> Redis perl >= 1.954 module.
> --
>
> I'm currently not using it on a system where the overhead is
> relevant for me, but
> i tried to reduce the number of redis command executed.
> I hope this will reduce the overhead significant.
that's great news. Thanks!
> Feedback and test results welcome.
I will, as soon as I have something to share!
p@rick
> Am 2015-07-15 23:22, schrieb Patrick Ben Koetter:
> >Markus,
> >
> >are you planning to add 'password' and 'database ID' support for redis
> >connects to RuleTimingRedis?
> >
> >What's your experience regarding Timing overhead? My simple tests
> >on the
> >commandlne show about 1 second overhead when RuleTimingRedis is added:
> >
> ># Without RuleTimingRedis
> >mail# time spamassassin --lint
> >
> >real 0m1.975s
> >user 0m1.852s
> >sys 0m0.116s
> >
> ># Enable RuleTimingRedis
> >mail# vim /etc/mail/spamassassin/init.pre
> >
> ># With RuleTimingRedis
> >mail# time spamassassin --lint
> >
> >real 0m2.828s
> >user 0m2.128s
> >sys 0m0.392s
> >
> >p@rick
> >
> >
> >
> >* Benning, Markus <ic...@markusbenning.de>:
> >>Hello,
> >>
> >>i want to announce the release of the SpamAssassin Plugins:
> >>
> >>RedisAWL - redis support for spamassassin AWL/TxRep
> >>RuleTimingRedis - collect SA rule timings in redis
> >>
> >>Both can be downloaded from CPAN or GitHub:
> >>
> >>https://metacpan.org/author/BENNING
> >>
> >>https://github.com/benningm
> >>
> >>Timings gathered with the RuleTimingRedis plugin can be used in
> >>collectd
> >>with the Collectd-Plugins-RedisClient module also available from CPAN.
> >>
> >> Markus
> >>
> >>--
> >>Markus Benning, https://markusbenning.de/
>
> --
> Markus Benning, https://markusbenning.de/
--
[*] sys4 AG
https://sys4.de, +49 (89) 30 90 46 64
Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis
Posted by Henrik K <he...@hege.li>.
On Tue, Feb 09, 2016 at 08:27:08AM +0100, Benning, Markus wrote:
> On 2016-02-05 16:25, Henrik K wrote:
> >You should start using the bundled Mail/SpamAssassin/Util/TinyRedis.pm
> >instead of cpan Redis module..
>
> I didn't notice theres a Redis module included with SpamAssassin.
> It seems like the module is undocumented.
>
> What does the SA Redis module different from the mainline module?
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=6972
It's made by Mark so quality is assured and it's always bad to rely on
external modules.
Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis
Posted by "Benning, Markus" <ic...@markusbenning.de>.
On 2016-02-05 16:25, Henrik K wrote:
> You should start using the bundled Mail/SpamAssassin/Util/TinyRedis.pm
> instead of cpan Redis module..
I didn't notice theres a Redis module included with SpamAssassin.
It seems like the module is undocumented.
What does the SA Redis module different from the mainline module?
Markus
--
https://markusbenning.de/
Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis
Posted by Henrik K <he...@hege.li>.
On Fri, Feb 05, 2016 at 03:17:21PM +0100, Benning, Markus wrote:
> On 2016-02-04 22:42, Skeffling wrote:
> >Sorry to reply to an old thread, but is it possible for the the
> >RedisAWL
> >plugin to specify a redis password too? (I'm looking to use TxRep
> >with a
> >redis backend...)
>
> Just released version 1.001:
>
> commit 4f7138e4823379f7728bcca005611d2a6640cf93
> Author: Markus Benning <ic...@markusbenning.de>
> Date: Fri Feb 5 15:11:45 2016 +0100
>
> Additional redis connection parameters
>
> The followin configuration parameters have been added:
>
> * auto_whitelist_redis_password
> * auto_whitelist_redis_database
> * auto_whitelist_redis_debug
You should start using the bundled Mail/SpamAssassin/Util/TinyRedis.pm
instead of cpan Redis module..
Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis
Posted by "Benning, Markus" <ic...@markusbenning.de>.
On 2016-02-04 22:42, Skeffling wrote:
> Sorry to reply to an old thread, but is it possible for the the
> RedisAWL
> plugin to specify a redis password too? (I'm looking to use TxRep with
> a
> redis backend...)
Just released version 1.001:
commit 4f7138e4823379f7728bcca005611d2a6640cf93
Author: Markus Benning <ic...@markusbenning.de>
Date: Fri Feb 5 15:11:45 2016 +0100
Additional redis connection parameters
The followin configuration parameters have been added:
* auto_whitelist_redis_password
* auto_whitelist_redis_database
* auto_whitelist_redis_debug
- Markus
--
https://markusbenning.de/
Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis
Posted by Skeffling <sk...@gmail.com>.
On 16/07/2015 19:58, Benning, Markus wrote:
> Hi Patrik,
>
> i just pushed Version 1.002 to github and CPAN:
>
> --
> The following new features have been added:
>
> - New option: timing_redis_password allows to specifiy a redis password
Sorry to reply to an old thread, but is it possible for the the RedisAWL
plugin to specify a redis password too? (I'm looking to use TxRep with a
redis backend...)
Thanks.
s.
Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis
Posted by "Benning, Markus" <ic...@markusbenning.de>.
Hi Patrik,
i just pushed Version 1.002 to github and CPAN:
--
The following new features have been added:
- New option: timing_redis_password allows to specifiy a redis
password
- New option: timing_redis_exclude_re excludes rules from timing
statistics. By default set to '^__' which will exclude all sub-rules
- New option: timing_redis_database allows to select a non-default
database in redis. (redis SWITCH call)
- New option: timing_redis_bulk_update will queue timing updates
before sending them to redis and execute them in a bulk via a
single call to a server-side script. By default this option is set
to 50 entries. Set to 0 do disable. Requires redis >= 2.6.0 and a
Redis perl >= 1.954 module.
--
I'm currently not using it on a system where the overhead is relevant
for me, but
i tried to reduce the number of redis command executed.
I hope this will reduce the overhead significant.
Feedback and test results welcome.
Markus
Am 2015-07-15 23:22, schrieb Patrick Ben Koetter:
> Markus,
>
> are you planning to add 'password' and 'database ID' support for redis
> connects to RuleTimingRedis?
>
> What's your experience regarding Timing overhead? My simple tests on
> the
> commandlne show about 1 second overhead when RuleTimingRedis is added:
>
> # Without RuleTimingRedis
> mail# time spamassassin --lint
>
> real 0m1.975s
> user 0m1.852s
> sys 0m0.116s
>
> # Enable RuleTimingRedis
> mail# vim /etc/mail/spamassassin/init.pre
>
> # With RuleTimingRedis
> mail# time spamassassin --lint
>
> real 0m2.828s
> user 0m2.128s
> sys 0m0.392s
>
> p@rick
>
>
>
> * Benning, Markus <ic...@markusbenning.de>:
>> Hello,
>>
>> i want to announce the release of the SpamAssassin Plugins:
>>
>> RedisAWL - redis support for spamassassin AWL/TxRep
>> RuleTimingRedis - collect SA rule timings in redis
>>
>> Both can be downloaded from CPAN or GitHub:
>>
>> https://metacpan.org/author/BENNING
>>
>> https://github.com/benningm
>>
>> Timings gathered with the RuleTimingRedis plugin can be used in
>> collectd
>> with the Collectd-Plugins-RedisClient module also available from CPAN.
>>
>> Markus
>>
>> --
>> Markus Benning, https://markusbenning.de/
--
Markus Benning, https://markusbenning.de/
Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis
Posted by Patrick Ben Koetter <p...@sys4.de>.
Markus,
are you planning to add 'password' and 'database ID' support for redis
connects to RuleTimingRedis?
What's your experience regarding Timing overhead? My simple tests on the
commandlne show about 1 second overhead when RuleTimingRedis is added:
# Without RuleTimingRedis
mail# time spamassassin --lint
real 0m1.975s
user 0m1.852s
sys 0m0.116s
# Enable RuleTimingRedis
mail# vim /etc/mail/spamassassin/init.pre
# With RuleTimingRedis
mail# time spamassassin --lint
real 0m2.828s
user 0m2.128s
sys 0m0.392s
p@rick
* Benning, Markus <ic...@markusbenning.de>:
> Hello,
>
> i want to announce the release of the SpamAssassin Plugins:
>
> RedisAWL - redis support for spamassassin AWL/TxRep
> RuleTimingRedis - collect SA rule timings in redis
>
> Both can be downloaded from CPAN or GitHub:
>
> https://metacpan.org/author/BENNING
>
> https://github.com/benningm
>
> Timings gathered with the RuleTimingRedis plugin can be used in collectd
> with the Collectd-Plugins-RedisClient module also available from CPAN.
>
> Markus
>
> --
> Markus Benning, https://markusbenning.de/
--
[*] sys4 AG
https://sys4.de, +49 (89) 30 90 46 64
Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis
Posted by Martin Gregorie <ma...@gregorie.org>.
On Tue, 2015-09-15 at 08:41 -0700, Ian Zimmerman wrote:
> On 2015-06-09 17:57 +0200, Benning, Markus wrote:
>
> > RuleTimingRedis - collect SA rule timings in redis
>
> I'm trying this out. I have a little annoying problem: the logs
> beginning on line 178 seem to go to stdout or stderr as well as
> syslog.
> The result is that cron sends me email every time spamd is restarted
> (after every rule update). Do you know how to change that? I find
> nothing about logging in perldoc Mail::SpamAssassin::Conf.
>
Assuming your sa_update script is the same as mine (I run Fedora Linux)
The obvious quick fix is to change line 2 in case 0) to read:
service spamassassin restart;;
and remove lines 1 and 3, which (in my script) are both 'echo' commands
reporting that the update completed and that Spamassassin was
restarted. If you're still seeing the unwanted messages, replace the
remaining line with:
service spamassassin restart 1>/dev/null 2>/dev.null;;
will will suppress anything that Spamassassin sends to stdout and
stderr. The sa_update script I have would still report updating errors
and the fact that there was no rules update because these are handled
by case 1, 2 and * and so can't be silenced by this edit.
> I suppose I could just delete those lines from the module :-) But
> then I would have extra work when I merge with any new versions you
> have.
>
Doing what I suggest would be easier than editing SA itself. Keep a
copy of your cron script when its doing what you want so, if an upgrade
slots in a new sa_update script that lets the unwanted text through
again, you can simply slot your modified version back in.
Martin
Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis
Posted by Ian Zimmerman <it...@buug.org>.
On 2015-06-09 17:57 +0200, Benning, Markus wrote:
> RuleTimingRedis - collect SA rule timings in redis
I'm trying this out. I have a little annoying problem: the logs
beginning on line 178 seem to go to stdout or stderr as well as syslog.
The result is that cron sends me email every time spamd is restarted
(after every rule update). Do you know how to change that? I find
nothing about logging in perldoc Mail::SpamAssassin::Conf.
I suppose I could just delete those lines from the module :-) But then
I would have extra work when I merge with any new versions you have.
Thanks for your ideas.
--
Please *no* private copies of mailing list or newsgroup messages.
Rule 420: All persons more than eight miles high to leave the court.