[Bug 3082] New: auto-whitelist database needs expiry

[bu...@bugzilla.spamassassin.org]

http://bugzilla.spamassassin.org/show_bug.cgi?id=3082

           Summary: auto-whitelist database needs expiry
           Product: Spamassassin
           Version: SVN Trunk (Latest Devel Version)
          Platform: Other
        OS/Version: other
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Libraries
        AssignedTo: spamassassin-dev@incubator.apache.org
        ReportedBy: quinlan@pathname.com


The auto-whitelist currently grows without bound.  Some form of expiry is
needed.

The current format of the DB contains the following two keys for each
email address:

  key => value
    user@example.com|ip=10.20 => 16
    user@example.com|ip=10.20|totscore => 19.41

  where
    user@example.com|ip=10.20 is a unique identifier for the user
    the first key/value pair is a message count
    the second key/value pair is the sum total score of those messages

Options to change DB format:

1. add a new key/value pair with a LRU mtime value or counter (atime is easiest)

   user@example.com|ip=10.20|time => 1077678476

   pros: easy, backward compatible
   cons: bloats DB, crufty, slow

2. change the DB format to be:

   user@example.com|ip=10.20 => [encoded field]

   where [encoded field] contains awl-version,atime,count,total (perhaps last N
   message scores instead of count,total)

   pros: clean, fast, extensible
   cons: requires conversion or switch

Options for conversion or switch:

1. Converting old DB to new DB format per-key in same file

   pros: conversion is spread out, no slow conversion or long locks
   cons: DB never shrinks, requires conversion code

2. Converting old DB to new DB format converting entire file at once

   pros: requires conversion code
   cons: DB can shrink, but may carry over more key/value pairs than will
         ultimately be used with atime-based expiry, so DB could end up larger
         than necessary

3. Just delete old DB and create a new DB

   pros: easiest
   cons: user loses AWL data



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.