You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Alex <my...@gmail.com> on 2011/11/30 06:17:58 UTC
Bayes database in mysql on multiple servers
Hi all,
I have two fedora15 boxes that process mail for a few domains, and
recently set up bayes in mysql for each of them. The servers are in
geographically different locations, a few hops from each other. Since
they both process mail for the same domains, I thought it made sense
to share the database between them.
What's the best way to do this? Set one as a master and the other as a
slave, or perhaps replication between them?
I also thought about something like drbd, but that seems a bit
excessive for just a database.
Thanks,
Alex
Re: Bayes database in mysql on multiple servers
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>>I have two fedora15 boxes that process mail for a few domains, and
>>recently set up bayes in mysql for each of them. The servers are in
>>geographically different locations, a few hops from each other. Since
>>they both process mail for the same domains, I thought it made sense
>>to share the database between them.
>>
>>What's the best way to do this? Set one as a master and the other as a
>>slave, or perhaps replication between them?
On 30.11.11 08:23, Michael Scheidell wrote:
>set master on mx1, slave on mx2.
>master is in charge of adding to db, and expiring, and slave can read it.
>problem: mx2 will get mostly spam, since spammers hit mx2 first, you
>'spam' hits will be lower then you thought.
well, in this case spamc on slave can preferrably query spamd on master first:
- no problem when master/mx1 is up
- minor performance delay when mx1/master is down
(even good to slow down spammers).
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
There's a long-standing bug relating to the x86 architecture that
allows you to install Windows. -- Matthew D. Fuller
Re: Bayes database in mysql on multiple servers
Posted by Michael Scheidell <mi...@secnap.com>.
On 12/1/11 10:06 AM, Benny Pedersen wrote:
> does not make sense so
hire a unix programmer to help you understand.
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation
* Best Mobile Solutions Product of 2011
* Best Intrusion Prevention Product
* Hot Company Finalist 2011
* Best Email Security Product
* Certified SNORT Integrator
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.spammertrap.com/
______________________________________________________________________
Re: Bayes database in mysql on multiple servers
Posted by Benny Pedersen <me...@junc.org>.
On Thu, 1 Dec 2011 01:01:04 -0500, Michael Scheidell wrote:
>>> sed -i '' -e '/INSERT INTO bayes_seen/s/INTO/IGNORE INTO/'
>>> MySQL.pm
>> ALTER TABLE `bayes_seen` ENGINE = INNODB
> no, that won't do anything (I use engine = innodb), what has innodb
> have to do with replication collisions?
innodb is more for single mysql server that have more then one client
update to same database, while replicating is more for more then one
client, its not simple, i just dont think sed is better
> nothing. nothing at all.
does not make sense so
Re: Bayes database in mysql on multiple servers
Posted by Michael Scheidell <mi...@secnap.com>.
> On Wed, 30 Nov 2011 08:23:59 -0500, Michael Scheidell wrote:
>
>> sed -i '' -e '/INSERT INTO bayes_seen/s/INTO/IGNORE INTO/' MySQL.pm
>>
>> (hey SA folks.. any reason not to just put that into 3.4.0? won't
>> hurt anything, will it?)
>
> or simply just
>
> ALTER TABLE `bayes_seen` ENGINE = INNODB
>
>
no, that won't do anything (I use engine = innodb), what has innodb have
to do with replication collisions?
nothing. nothing at all.
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation
* Best Mobile Solutions Product of 2011
* Best Intrusion Prevention Product
* Hot Company Finalist 2011
* Best Email Security Product
* Certified SNORT Integrator
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.spammertrap.com/
______________________________________________________________________
Re: Bayes database in mysql on multiple servers
Posted by Benny Pedersen <me...@junc.org>.
On Wed, 30 Nov 2011 08:23:59 -0500, Michael Scheidell wrote:
> sed -i '' -e '/INSERT INTO bayes_seen/s/INTO/IGNORE INTO/'
> MySQL.pm
>
> (hey SA folks.. any reason not to just put that into 3.4.0? won't
> hurt anything, will it?)
or simply just
ALTER TABLE `bayes_seen` ENGINE = INNODB
Re: Bayes database in mysql on multiple servers
Posted by Michael Scheidell <mi...@secnap.com>.
> Hi all,
>
> I have two fedora15 boxes that process mail for a few domains, and
> recently set up bayes in mysql for each of them. The servers are in
> geographically different locations, a few hops from each other. Since
> they both process mail for the same domains, I thought it made sense
> to share the database between them.
>
> What's the best way to do this? Set one as a master and the other as a
> slave, or perhaps replication between them?
>
easy:
set master on mx1, slave on mx2.
master is in charge of adding to db, and expiring, and slave can read it.
problem: mx2 will get mostly spam, since spammers hit mx2 first, you
'spam' hits will be lower then you thought.
hard:
master/master.
you have replication issues, especially when the sam spammer sends 500
emails to the mx1, and the same 500 to mx2.
only run manual expire via cronjob on master.
try this patch: (changes insert into bayes_seen to insert ignore into,
ymmv, use at own risk, your HP printer sets on fire because of it, its
not my fault)
cd /usr/local/lib/perl5/site_perl/${pv}/Mail/SpamAssassin/BayesStore
sed -i '' -e '/INSERT INTO bayes_seen/s/INTO/IGNORE INTO/' MySQL.pm
(hey SA folks.. any reason not to just put that into 3.4.0? won't hurt
anything, will it?)
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation
* Best Mobile Solutions Product of 2011
* Best Intrusion Prevention Product
* Hot Company Finalist 2011
* Best Email Security Product
* Certified SNORT Integrator
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.spammertrap.com/
______________________________________________________________________
Re: Bayes database in mysql on multiple servers
Posted by Walter Hurry <wa...@lavabit.com>.
On Wed, 30 Nov 2011 08:25:43 -0800, John Hardin wrote:
> On Wed, 30 Nov 2011, Walter Hurry wrote:
>> Why replicate? Why not just share the same database?
>
> Latency and reliability of the link between the "geographically separate
> locations". Replication is typically robust in the face of unreliable
> networking and high latency and will recover from outages, and most
> people don't want their mail delivery system doing WAN database queries
> with the associated (assumed) high latency, limited bandwidth and risk
> of service interruption that is inherent in a WAN.
Good points. Actually I missed the "geographically separate locations"
bit - mea culpa.
Re: Bayes database in mysql on multiple servers
Posted by John Hardin <jh...@impsec.org>.
On Wed, 30 Nov 2011, Walter Hurry wrote:
> On Wed, 30 Nov 2011 09:11:49 +0100, Robert Schetterer wrote:
>
>> Am 30.11.2011 09:06, schrieb Matus UHLAR - fantomas:
>>> On 30.11.11 00:17, Alex wrote:
>>>> I have two fedora15 boxes that process mail for a few domains, and
>>>> recently set up bayes in mysql for each of them. The servers are in
>>>> geographically different locations, a few hops from each other. Since
>>>> they both process mail for the same domains, I thought it made sense
>>>> to share the database between them.
>>>>
>>>> What's the best way to do this? Set one as a master and the other as a
>>>> slave, or perhaps replication between them?
>>>>
>>>> I also thought about something like drbd, but that seems a bit
>>>> excessive for just a database.
>>
>> dont use drbd with mysql store, you dont need it
>>>
>>> I think this is question for MySQL mailing list, not for SA.
>>>
>> you can use i.e master-master replication ( which i do ), but be aware
>> you might get doubles with bayes store, this should be ignored
>>
>> but i am told PostgreSQL is better in replacation stuff
>
> Why replicate? Why not just share the same database?
Latency and reliability of the link between the "geographically separate
locations". Replication is typically robust in the face of unreliable
networking and high latency and will recover from outages, and most people
don't want their mail delivery system doing WAN database queries with the
associated (assumed) high latency, limited bandwidth and risk of
service interruption that is inherent in a WAN.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
There is no better measure of the unthinking contempt of the
environmentalist movement for civilization than their call to turn
off the lights and sit in the dark. -- Sultan Knish
-----------------------------------------------------------------------
15 days until Bill of Rights day
Re: Bayes database in mysql on multiple servers
Posted by Nigel Frankcom <ni...@blue-canoe.com>.
On Wed, 30 Nov 2011 15:14:33 +0000 (UTC), Walter Hurry
<wa...@lavabit.com> wrote:
>On Wed, 30 Nov 2011 09:11:49 +0100, Robert Schetterer wrote:
>
>> Am 30.11.2011 09:06, schrieb Matus UHLAR - fantomas:
>>> On 30.11.11 00:17, Alex wrote:
>>>> I have two fedora15 boxes that process mail for a few domains, and
>>>> recently set up bayes in mysql for each of them. The servers are in
>>>> geographically different locations, a few hops from each other. Since
>>>> they both process mail for the same domains, I thought it made sense
>>>> to share the database between them.
>>>>
>>>> What's the best way to do this? Set one as a master and the other as a
>>>> slave, or perhaps replication between them?
>>>>
>>>> I also thought about something like drbd, but that seems a bit
>>>> excessive for just a database.
>>
>> dont use drbd with mysql store, you dont need it
>>>
>>> I think this is question for MySQL mailing list, not for SA.
>>>
>> you can use i.e master-master replication ( which i do ), but be aware
>> you might get doubles with bayes store, this should be ignored
>>
>> but i am told PostgreSQL is better in replacation stuff
>
>Why replicate? Why not just share the same database?
No failover with shared. Distributed adds redundancy.
KR
Nigel
Re: Bayes database in mysql on multiple servers
Posted by Walter Hurry <wa...@lavabit.com>.
On Wed, 30 Nov 2011 09:11:49 +0100, Robert Schetterer wrote:
> Am 30.11.2011 09:06, schrieb Matus UHLAR - fantomas:
>> On 30.11.11 00:17, Alex wrote:
>>> I have two fedora15 boxes that process mail for a few domains, and
>>> recently set up bayes in mysql for each of them. The servers are in
>>> geographically different locations, a few hops from each other. Since
>>> they both process mail for the same domains, I thought it made sense
>>> to share the database between them.
>>>
>>> What's the best way to do this? Set one as a master and the other as a
>>> slave, or perhaps replication between them?
>>>
>>> I also thought about something like drbd, but that seems a bit
>>> excessive for just a database.
>
> dont use drbd with mysql store, you dont need it
>>
>> I think this is question for MySQL mailing list, not for SA.
>>
> you can use i.e master-master replication ( which i do ), but be aware
> you might get doubles with bayes store, this should be ignored
>
> but i am told PostgreSQL is better in replacation stuff
Why replicate? Why not just share the same database?
Re: Bayes database in mysql on multiple servers
Posted by Robert Schetterer <ro...@schetterer.org>.
Am 30.11.2011 09:06, schrieb Matus UHLAR - fantomas:
> On 30.11.11 00:17, Alex wrote:
>> I have two fedora15 boxes that process mail for a few domains, and
>> recently set up bayes in mysql for each of them. The servers are in
>> geographically different locations, a few hops from each other. Since
>> they both process mail for the same domains, I thought it made sense
>> to share the database between them.
>>
>> What's the best way to do this? Set one as a master and the other as a
>> slave, or perhaps replication between them?
>>
>> I also thought about something like drbd, but that seems a bit
>> excessive for just a database.
dont use drbd with mysql store, you dont need it
>
> I think this is question for MySQL mailing list, not for SA.
>
you can use i.e master-master replication ( which i do ), but be aware
you might get doubles with bayes store, this should be ignored
but i am told PostgreSQL is better in replacation stuff
--
Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
Re: Bayes database in mysql on multiple servers
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 30.11.11 00:17, Alex wrote:
>I have two fedora15 boxes that process mail for a few domains, and
>recently set up bayes in mysql for each of them. The servers are in
>geographically different locations, a few hops from each other. Since
>they both process mail for the same domains, I thought it made sense
>to share the database between them.
>
>What's the best way to do this? Set one as a master and the other as a
>slave, or perhaps replication between them?
>
>I also thought about something like drbd, but that seems a bit
>excessive for just a database.
I think this is question for MySQL mailing list, not for SA.
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Support bacteria - they're the only culture some people have.