How best to validate a BayesStore implementation?

[Michael Alan Dorman <md...@ironicdesign.com>]

Hey all,

I've been writing a BerkeleyDB-based BayesStore (with the intention of
contributing it when it's complete), and I was wondering if the regular
SA test suite is sufficient to validate the implementation, or if
someone has any additional materials that would be helpful?

Most of the code, is, as one might expect, from the DBM implementation,
but I'm using more advanced BDB capabilities to do things like keep an
index on atime to speed expiration, and using BDB's transactional
capabilities to avoid having to journal, etc., and, of course, those
are the bits that are most fundamental, and therefore most in need of
testing.

Thanks for any guidance,

Mike.

Re: How best to validate a BayesStore implementation?

[Justin Mason <jm...@gmail.com>]

On Thu, Dec 18, 2008 at 01:41, Mark Martinec <Ma...@ijs.si> wrote:
>> Is there any need for me to bother with SVN, at least initially?
>> It seems to me that working against 3.2.5 should be adequate,
>> but I don't pretend to know for certain.
>
> 3.2.5 should be alright for the purpose, although generally new development
> would be least painful when starting from current code. As it stands
> presently, the CVS head is stable and is still compatible with 3.2.5,
> so one can switch back and forth if necessary. One of the shifts in 3.3
> is tighter error handling and informative reporting/logging of all
> unexpected/unhandled error conditions, please keep it in mind.

and, of course, if you want the plugin to get into released versions
of SA, it'll _need_ to be made against trunk for that to happen ;)

--j.

Re: How best to validate a BayesStore implementation?

[Michael Alan Dorman <md...@ironicdesign.com>]

On Thu, 18 Dec 2008 02:41:41 +0100
Mark Martinec <Ma...@ijs.si> wrote:

Hi, Mark,

> tieing? When already choosing BerkeleyDB for its features, my
> preferred choice would be to use explicit bdb function calls, which
> give you full access to options and first-hand error status on
> failures. Hiding bdb API under a perl 'tie' limits your choices and
> muddies failure reasons.

Sorry, I referred to it as tieing because that's how the docs refer
to it and it's rife in the method names, but I actually use explicit BDB
calls.

> 3.2.5 should be alright for the purpose, although generally new
> development would be least painful when starting from current code.

Sure.

> As it stands presently, the CVS head is stable and is still
> compatible with 3.2.5, so one can switch back and forth if necessary.
> One of the shifts in 3.3 is tighter error handling and informative
> reporting/logging of all unexpected/unhandled error conditions,
> please keep it in mind.

I try to be diligent, but everybody fails now and again.  I have no
doubt that more eyes on the code will spot those quickly. :)

Mike.

Re: How best to validate a BayesStore implementation?

[Mark Martinec <Ma...@ijs.si>]

Mike,

> I suspect that there will have to be some discussion of some particular
> choices I made--for instance, if they're tieing in R/O mode, I will
> just return 0 if _any_ tables are missing, stuff like that.  So it's
> maybe stricter in some ways.

tieing? When already choosing BerkeleyDB for its features, my preferred
choice would be to use explicit bdb function calls, which give you full access
to options and first-hand error status on failures. Hiding bdb API under a
perl 'tie' limits your choices and muddies failure reasons.

> Is there any need for me to bother with SVN, at least initially?
> It seems to me that working against 3.2.5 should be adequate,
> but I don't pretend to know for certain.

3.2.5 should be alright for the purpose, although generally new development
would be least painful when starting from current code. As it stands
presently, the CVS head is stable and is still compatible with 3.2.5,
so one can switch back and forth if necessary. One of the shifts in 3.3
is tighter error handling and informative reporting/logging of all
unexpected/unhandled error conditions, please keep it in mind.

  Mark

Re: How best to validate a BayesStore implementation?

[Justin Mason <jm...@gmail.com>]

it'd also be kick-ass if this plugin could auto-upgrade when the
BerkeleyDB module
is upgraded:
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=4416 ;)

re coding style: http://wiki.apache.org/spamassassin/CodingStyle

--j.

On Tue, Dec 16, 2008 at 16:36, Michael Alan Dorman
<md...@ironicdesign.com> wrote:
> On Tue, 16 Dec 2008 09:56:04 -0600
> Michael Parker <pa...@pobox.com> wrote:
>
>> Copying the t-tests for your particular implementation is a good
>> start.  You may need to adjust them a bit and feel free to expand
>> them to test even more functionality.
>
> Sure.  Is there any need for me to bother with SVN, at least initially?
> It seems to me that working against 3.2.5 should be adequate, but I
> don't pretend to know for certain.
>
>> Then you have the masses/bayes-testing directory that will give you
>> some help.  Inside that directory is the benchmark tool that you can
>> use to test the speed of your implementation against others.
>
> Ah, this I probably wouldn't have noticed right away, which would have
> been a shame since what I really want is for this to be screamingly
> absurdly fast.
>
>> Looking forward to seeing what you've come up with.
>
> I've got everything done except restore_database at this point, but
> it's been about six hours of coding and visual inspection of same;
> hence the need for some good test code. :)
>
> I suspect that there will have to be some discussion of some particular
> choices I made--for instance, if they're tieing in R/O mode, I will
> just return 0 if _any_ tables are missing, stuff like that.  So it's
> maybe stricter in some ways.
>
> Also, is there a particular style guide that SA follows?  Maybe with a
> perltidy profile I can use to reformat to it?
>
> Mike.
>
>

Re: How best to validate a BayesStore implementation?

[Michael Alan Dorman <md...@ironicdesign.com>]

On Tue, 16 Dec 2008 09:56:04 -0600
Michael Parker <pa...@pobox.com> wrote:

> Copying the t-tests for your particular implementation is a good  
> start.  You may need to adjust them a bit and feel free to expand
> them to test even more functionality.

Sure.  Is there any need for me to bother with SVN, at least initially?
It seems to me that working against 3.2.5 should be adequate, but I
don't pretend to know for certain.

> Then you have the masses/bayes-testing directory that will give you  
> some help.  Inside that directory is the benchmark tool that you can  
> use to test the speed of your implementation against others.

Ah, this I probably wouldn't have noticed right away, which would have
been a shame since what I really want is for this to be screamingly
absurdly fast.

> Looking forward to seeing what you've come up with.

I've got everything done except restore_database at this point, but
it's been about six hours of coding and visual inspection of same;
hence the need for some good test code. :)

I suspect that there will have to be some discussion of some particular
choices I made--for instance, if they're tieing in R/O mode, I will
just return 0 if _any_ tables are missing, stuff like that.  So it's
maybe stricter in some ways.

Also, is there a particular style guide that SA follows?  Maybe with a
perltidy profile I can use to reformat to it?

Mike.

Re: How best to validate a BayesStore implementation?

[Michael Parker <pa...@pobox.com>]

On Dec 16, 2008, at 8:09 AM, Michael Alan Dorman wrote:

> Hey all,
>
> I've been writing a BerkeleyDB-based BayesStore (with the intention of
> contributing it when it's complete), and I was wondering if the  
> regular
> SA test suite is sufficient to validate the implementation, or if
> someone has any additional materials that would be helpful?
>
> Most of the code, is, as one might expect, from the DBM  
> implementation,
> but I'm using more advanced BDB capabilities to do things like keep an
> index on atime to speed expiration, and using BDB's transactional
> capabilities to avoid having to journal, etc., and, of course, those
> are the bits that are most fundamental, and therefore most in need of
> testing.
>
> Thanks for any guidance,

Copying the t-tests for your particular implementation is a good  
start.  You may need to adjust them a bit and feel free to expand them  
to test even more functionality.

Then you have the masses/bayes-testing directory that will give you  
some help.  Inside that directory is the benchmark tool that you can  
use to test the speed of your implementation against others.

Looking forward to seeing what you've come up with.

Michael