Bayes DB on single-node MySQL cluster

[Paul Hirose <pt...@ucdavis.edu>]

RHEL5.5, MySQL GA 5.0.77, MySQL Cluster 7.1.4b, 64bit, SpamAssassin 3.2.5 (but hoping to go to 3.3.1 soon.)

In short, I stumbled across: http://www.clusterdb.com/mysql-cluster/how-can-a-database-be-in-memory-and-durable-at-the-same-time/ which essentially shows how to create a MySQL Cluster, but of only one node.  This gets me an all-in-memory database *and* row-level locking.  Sorta the best of both worlds, compared to using Heap/Memory vs InnoDB engine.  Has anyone tried this, and  did it work for you?

There've been threads against using master/master replication or cluster, and a couple bugzilla entries specifically state cluster/replication is "unsafe".  I think the main reason behind this is simply the duplication of data, and clear example was given in one bugzilla report.  But if I do a single-node cluster (only one data/MySQL node), then there are no copies of data.  Thus, it can't get out of sync, because there's nothing else to get out of sync with. Would this then be "safe"?  Or is there something inherent in the clustering/replication that just doesn't work?

Given this, I know there are folks using m/m-replication, and have seen reference to various threads.  So far, I haven't see anyone post a glaring example about how it failed or anything, but I'm still a touch shy about going against the devs :)  

Thank you,
PH

PS
I realize I can essentially get all in-memory for InnoDB w/a high-enough innodb_buffer_pool_size.  But we keep hitting some wall after which performance just drops off a cliff.  Haven't determined why yet, and while we investigate, was also thinking of other ways to implement the DB, rather than just constantly tweaking a couple variables and seeing what happens.  It's not RAM (plenty) and it's not IO (which there shouldn't be much of anyway given the DB is all in RAM and there's nothing else running on this system.)  It's just an odd mystery to me.

==
Paul Hirose
pthirose@ucdavis.edu

Re: Bayes DB on single-node MySQL cluster

[Michael Scheidell <sc...@secnap.net>]

On 7/26/10 5:02 PM, Paul Hirose wrote:
> RHEL5.5, MySQL GA 5.0.77, MySQL Cluster 7.1.4b, 64bit, SpamAssassin 3.2.5 (but hoping to go to 3.3.1 soon.)
>
> In short, I stumbled across: http://www.clusterdb.com/mysql-cluster/how-can-a-database-be-in-memory-and-durable-at-the-same-time/ which essentially shows how to create a MySQL Cluster, but of only one node.  This gets me an all-in-memory database *and* row-level locking.  Sorta the best of both worlds, compared to using Heap/Memory vs InnoDB engine.  Has anyone tried this, and  did it work for you?
>
>    
and if you have a 3.5GB bayes database, don't you need 3.5GB ram?

where is that bugzilla report?  I might have a solution for it.

> There've been threads against using master/master replication or cluster, and a couple bugzilla entries specifically state cluster/replication is "unsafe".  I think the main reason behind this is simply the duplication of data, and clear example was given in one bugzilla report.  But if I do a single-node cluster (only one data/MySQL node), then there are no copies of data.  Thus, it can't get out of sync, because there's nothing else to get out of sync with. Would this then be "safe"?  Or is there something inherent in the clustering/replication that just doesn't work?
>
> Given this, I know there are folks using m/m-replication, and have seen reference to various threads.  So far, I haven't see anyone post a glaring example about how it failed or anything, but I'm still a touch shy about going against the devs :)
>    
biggest issues seem to be, you need a 5.1.47 or newer mysql, and I think 
you want to use the plugin (i think).
still get deadlocks while multi threads are trying to update the bayes 
DB.  but if you 'swatch' it, maybe you just retry?
or, heck, its just bayes, who care? the spammers will hit you again (and 
if you got the deadlock, they did)

-- 
Michael Scheidell, CTO
Phone: 561-999-5000, x 1259
 > *| *SECNAP Network Security Corporation

    * Certified SNORT Integrator
    * 2008-9 Hot Company Award Winner, World Executive Alliance
    * Five-Star Partner Program 2009, VARBusiness
    * Best in Email Security,2010: Network Products Guide
    * King of Spam Filters, SC Magazine 2008


______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r). 
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________