read_scoreonly_config - how to clear previous user settings?? (repost)

[Per Jessen <pe...@computer.org>]

I posted this to spamassassin-general, but I guess that wasn't quite the
right forum.

I'm using read_scoreonly_config to load new user-settings in an SA daemon
process (spampd).  Following that I do a signal_user_changed(). 
However, how do I clear out the score settings from the previous
read_scoreonly_config() ? For instance, I have a user that uses "score
MICROSOFT_EXECUTABLE 10", whereas another one doesn't - yet when the scoring
rules for the latter is loaded, the "MICROSOFT_EXECUTABLE 10" is still in
effect from the previous one. 
(Note - mysql is not involved at all.)


/Per Jessen, Zurich

Re: read_scoreonly_config - how to clear previous user settings??

[Per Jessen <pe...@computer.org>]

Per Jessen wrote:
> I have yet to study the performance-impact in thawing a Conf object, but
> given that I really need the correct per-user behaviour, it's not overly
> important.

Just a quick follow-up - instead of freeze/thaw I tried using store
retrieve, which cuts down on the memory footprint.  
The file is written once and then read many times by a number of children,
and I can just let the file-system cache it as it sees fit. 
Alternatively, I tried writing the file to /dev/shm/, but I haven't seen any
major differences when compared to writing to disk.


/Per Jessen, Zurich

Re: read_scoreonly_config - how to clear previous user settings??

[Per Jessen <pe...@computer.org>]

Theo Van Dinter wrote:

> In my tests, freeze/thaw seems to work nicely, and I've implemented it in
> my new spamd code.  I haven't quite figured out where it'll fit in an API.
> If it goes into the M::SA modules, it'll be a generic requirement, which
> I don't want.  But leaving it in spamd by itself means that third-party
> daemons have to implement it for themselves. :(

Helped along by Theos spamd changes, I've implemented the proposed freeze
thaw in spampd and it works here too. Seems to use quite a bit of memory
though, looks like 4-5M.  
I have yet to study the performance-impact in thawing a Conf object, but
given that I really need the correct per-user behaviour, it's not overly
important.  
Many thanks for the help, guys!


/Per Jessen, Zurich

Re: read_scoreonly_config - how to clear previous user settings?? (repost)

[Theo Van Dinter <fe...@kluge.net>]

On Tue, Apr 13, 2004 at 10:57:24PM +0200, Per Jessen wrote:
> > Doing a little research, looks like Storable::freeze()/thaw() might make
> > a good backup/restore utility, but I'll have to look into it some more.
> 
> I'd be very interested to see what you can work out.  

In my tests, freeze/thaw seems to work nicely, and I've implemented it in
my new spamd code.  I haven't quite figured out where it'll fit in an API.
If it goes into the M::SA modules, it'll be a generic requirement, which
I don't want.  But leaving it in spamd by itself means that third-party
daemons have to implement it for themselves. :(

-- 
Randomly Generated Tagline:
"Unfortunately, we keep kicking ourselves in the foot." - RAY WILKINS, BBC1

Re: read_scoreonly_config - how to clear previous user settings?? (repost)

[Per Jessen <pe...@computer.org>]

Theo Van Dinter wrote:

> Been looking at things a little more: read_scoreonly_config() does
> more than scores...  it's scores, whitelists, blacklists, version tags,
> required_hits, rewrite_header, etc.  Basically anything the users are
> allowed to change.

Yeah, I got that far too - the 'scoreonly' bit seems to be a slight
misnomer :-(

> 2) somehow make a complete duplicate of $m->{conf} values.  something
> like:
> 
>   $m = new Mail::SpamAssassin(...);
>   $conf = $m->backup_config();
>   ... do stuff ...
>   $m->restore_config($conf);
>   $m->read_scoreonly_config(...);

Which is pretty close to the kind of behaviour I was expecting.

> Doing a little research, looks like Storable::freeze()/thaw() might make
> a good backup/restore utility, but I'll have to look into it some more.

I'd be very interested to see what you can work out.  


/Per Jessen, Zurich

Re: read_scoreonly_config - how to clear previous user settings?? (repost)

[Theo Van Dinter <fe...@kluge.net>]

On Tue, Apr 13, 2004 at 02:53:35PM -0400, Theo Van Dinter wrote:
> the exact same issue you're talking about.  My initial reaction is to
> cheat and save off a copy of the scores arrays in the daemon, and copy it
> into place per user switch, but that's not the right way to do it (tm). :(

Turns out that doesn't really solve the problem anyway...

Been looking at things a little more: read_scoreonly_config() does
more than scores...  it's scores, whitelists, blacklists, version tags,
required_hits, rewrite_header, etc.  Basically anything the users are
allowed to change.

So there are two options that I can see:

1) kill $M::SA->{conf}, and re-init ala:

  $m = new Mail::SpamAssassin(...);
  ... do stuff ...
  $m->{conf} = new Mail::SpamAssassin::Conf ($m);
  $m->read_scoreonly_config(...);

This would solve the problem completely, but would cause each message
to load all of the configs, which kills most of the benefits of running
in a daemon in the first place.

2) somehow make a complete duplicate of $m->{conf} values.  something
like:

  $m = new Mail::SpamAssassin(...);
  $conf = $m->backup_config();
  ... do stuff ...
  $m->restore_config($conf);
  $m->read_scoreonly_config(...);

where backup_config and restore_config would do a complete recursive copy
of the object's variables.  it needs to be completely recursive since
things like the scores array is really an array of hash references,
and we need to dereference everything.

Doing a little research, looks like Storable::freeze()/thaw() might make
a good backup/restore utility, but I'll have to look into it some more.

-- 
Randomly Generated Tagline:
And shun the frumious Bandersnatch.

Re: read_scoreonly_config - how to clear previous user settings?? (repost)

[Theo Van Dinter <fe...@kluge.net>]

On Tue, Apr 13, 2004 at 08:34:41PM +0200, Per Jessen wrote:
> As a general question, when people use spamd or spampd (as I do) with
> per-user settings & black/whitelists, how do they sort this little problem
> out?  Or work around it for that matter? 

Well, for spamd it's pretty easy.  The current model forks a new child
per incoming message, so the configuration only has to survive for
that child, which makes lots of things much easier.

The 3.0 version (assuming I can get it all done) is going to prefork
and deal with multiple users per child, which is going to run me into
the exact same issue you're talking about.  My initial reaction is to
cheat and save off a copy of the scores arrays in the daemon, and copy it
into place per user switch, but that's not the right way to do it (tm). :(

-- 
Randomly Generated Tagline:
"See, plastic and fats have certain molecular similarities, which is why
 it's so hard to get say ... bacon fat off of a bicycle seat."
         - Alton Brown, Good Eats, "Salad Daze"

Re: read_scoreonly_config - how to clear previous user settings?? (repost)

[Per Jessen <pe...@computer.org>]

Justin Mason wrote:

> Per Jessen writes:
>> I posted this to spamassassin-general, but I guess that wasn't quite the
>> right forum.
>> 
>> I'm using read_scoreonly_config to load new user-settings in an SA daemon
>> process (spampd).  Following that I do a signal_user_changed().
>> However, how do I clear out the score settings from the previous
>> read_scoreonly_config() ? For instance, I have a user that uses "score
>> MICROSOFT_EXECUTABLE 10", whereas another one doesn't - yet when the
>> scoring rules for the latter is loaded, the "MICROSOFT_EXECUTABLE 10" is
>> still in effect from the previous one.
>> (Note - mysql is not involved at all.)
> 
> Hi Per --
> 
> currently, there's no way to do this :(   The Conf object stays the
> same between scans, so prior settings remain active.

Hi Justin,

I was afraid of that ... and re-initialising the Conf object per user would
presumably be a major performance hit. Although I guess I could load all
the default scores again without causing to much upheaval. 

As a general question, when people use spamd or spampd (as I do) with
per-user settings & black/whitelists, how do they sort this little problem
out?  Or work around it for that matter? 


/Per Jessen, Zurich