You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-user@james.apache.org by "John G. Norman" <jo...@7fff.com> on 2007/03/08 15:39:31 UTC
Multiple James instances can use database repository?
Hi.
I'm thinking a bit about failover and James.
If I'm running two Jameses simultaneously, one on each of two systems, can
they point to the same database-backed repositories (
http://james.apache.org/server/2.3.0/repositories.html)?
One would be primary MX, the other secondary. So typically, one would
probably be handling most of the load, though doubtless some senders would
hit the secondary for the usual variety of reasons.
The load is significant, but not massive; I only use James for bounce
control (everything else on Postfix).
I saw some ancient discussion such as this:
http://marc.theaimsgroup.com/?l=james-user&m=100434648513598&w=2
John
RE: [2] Multiple James instances can use database repository?
Posted by "Noel J. Bergman" <no...@devtech.com>.
Stefano Bagnara wrote:
> our MailRepositories and SpoolRepositories (both file and
> jdbc specializations) [do] Locking their own way.
And caching.
In any event, there are plans to permit clustering, and for the time being,
your recommendations (in other messages) are appropriate.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: [2] Multiple James instances can use database repository?
Posted by Stefano Bagnara <ap...@bago.org>.
As I told, our MailRepositories and SpoolRepositories (both file and
jdbc specializations) needs Locking their own way. The locking provided
by the filesystem or by sql server will not work.
Basically James "accept" a message from the spool (read it, and LOCK it
in the application), then it process it (the message is still in the
original table) and when the processing is finisced it deletes the
message or at least UNLOCK it.
In the mean time others threads of the same application will ignore that
message because of the LOCK we have in memory. If the threads have no
access to the locks table then the same message could be processed by 2
james at the same time: both will process it, one of them will delete it
and the other will throw an error when it will try to delete it and it
is already been deleted. But in the end the message will be processed TWICE.
Maybe we should add this to the FAQs: but you should take care that when
an application does not specifically document that it support clustering
this way, most time it won't work.
Workarounds?
1) distributed locking (see james-ha on sourceforge as an example,
because it is outdated).
2) use of SMTP primary/secondary MX servers, without shared tables
between the servers.
3) use at least separate SpoolRepositories and share only
MailRepositories (I'm not sure but this should work), then you write a
script to move mails from one spool to the other when one of the servers
goes down.
Stefano
sunsetartproducts ha scritto:
> I use this set-up for the whitelist and spam analysis with 2 instances of
> James 2.3.0 on different servers sharing a MySql database on a third server.
> One machine is the fallover machine the other the primary mail server. I
> have not tried using the database to hold the mails, just the whitelist and
> spam database. But at least in theory Mysql would allow access by two
> different instances, as the file locking in MySQL is at the database level.
>
> When I am using it in this way, I can add or remove data from the database
> from either mail server, MySQL just waits to finish the first process before
> committing to the next. (not much of a wait as the processes take only a
> millisecond or less.) Of course the servers are only 10 feet apart and have
> only one hop on a switch to navigate.
>
> If you are using a file system repository for the mails data instead of a
> SQL Server database, then you would run into the problem of file locking if
> the two servers were trying to access the same files on the hard drive at
> the same time.
>
>
> John G. Norman-2 wrote:
>> Stephano,
>>
>> THANKS.
>>
>> John
>>
>> On 3/8/07, Stefano Bagnara <ap...@bago.org> wrote:
>>> You can't use the same db table for 2 different repositories instances
>>> because the locking is at an application level. You would need to
>>> rewrite a distributed locking support (someone did something about this
>>> in past: see james-ha project on sourceforge).
>>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: [2] Multiple James instances can use database repository?
Posted by Stefano Bagnara <ap...@bago.org>.
This is the opposite of what the topic is about.
Btw I never tried it, but it should work as IIRC we don't use
autoincrement tables (that are the main problem with C-JDBC).
Stefano
Andy Dennis ha scritto:
> Has anyone tried using the Sequoia project (ex C-JDBC) for a clustered
> database backend solution? This would give you the ability to duplicate
> your backend database if it worked well.
>
> I must admit that I haven't worked with James for a while, but as long
> as you can used a different jdnc driver it could posibly work. If you
> are using mysql as a back end database it will be happy with mysql SQL
> semantics.
> Regards,
> Andy.
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: [2] Multiple James instances can use database repository?
Posted by Andy Dennis <an...@btinternet.com>.
Has anyone tried using the Sequoia project (ex C-JDBC) for a clustered
database backend solution? This would give you the ability to duplicate
your backend database if it worked well.
I must admit that I haven't worked with James for a while, but as long as
you can used a different jdnc driver it could posibly work. If you are
using mysql as a back end database it will be happy with mysql SQL
semantics.
Regards,
Andy.
----- Original Message -----
From: "sunsetartproducts" <ha...@yahoo.com>
To: <se...@james.apache.org>
Sent: Saturday, March 10, 2007 2:54 AM
Subject: Re: [2] Multiple James instances can use database repository?
>
> I use this set-up for the whitelist and spam analysis with 2 instances of
> James 2.3.0 on different servers sharing a MySql database on a third
> server.
> One machine is the fallover machine the other the primary mail server. I
> have not tried using the database to hold the mails, just the whitelist
> and
> spam database. But at least in theory Mysql would allow access by two
> different instances, as the file locking in MySQL is at the database
> level.
>
> When I am using it in this way, I can add or remove data from the database
> from either mail server, MySQL just waits to finish the first process
> before
> committing to the next. (not much of a wait as the processes take only a
> millisecond or less.) Of course the servers are only 10 feet apart and
> have
> only one hop on a switch to navigate.
>
> If you are using a file system repository for the mails data instead of a
> SQL Server database, then you would run into the problem of file locking
> if
> the two servers were trying to access the same files on the hard drive at
> the same time.
>
>
> John G. Norman-2 wrote:
>>
>> Stephano,
>>
>> THANKS.
>>
>> John
>>
>> On 3/8/07, Stefano Bagnara <ap...@bago.org> wrote:
>>>
>>> You can't use the same db table for 2 different repositories instances
>>> because the locking is at an application level. You would need to
>>> rewrite a distributed locking support (someone did something about this
>>> in past: see james-ha project on sourceforge).
>>>
>>
>>
>
> --
> View this message in context:
> http://www.nabble.com/Multiple-James-instances-can-use-database-repository--tf3369616.html#a9406721
> Sent from the James - Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
> For additional commands, e-mail: server-user-help@james.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: [2] Multiple James instances can use database repository?
Posted by "John G. Norman" <jo...@7fff.com>.
Wow, I didn't see that this thread had continued.
It would truly be great if some of these existing scenarios / strategies
were added to the Wiki and/or the FAQ.
On 3/9/07, sunsetartproducts <ha...@yahoo.com> wrote:
>
>
> I use this set-up for the whitelist and spam analysis with 2 instances of
> James 2.3.0 on different servers sharing a MySql database on a third
> server.
> One machine is the fallover machine the other the primary mail server. I
> have not tried using the database to hold the mails, just the whitelist
> and
> spam database. But at least in theory Mysql would allow access by two
> different instances, as the file locking in MySQL is at the database
> level.
>
> When I am using it in this way, I can add or remove data from the database
> from either mail server, MySQL just waits to finish the first process
> before
> committing to the next. (not much of a wait as the processes take only a
> millisecond or less.) Of course the servers are only 10 feet apart and
> have
> only one hop on a switch to navigate.
>
> If you are using a file system repository for the mails data instead of a
> SQL Server database, then you would run into the problem of file locking
> if
> the two servers were trying to access the same files on the hard drive at
> the same time.
>
>
> John G. Norman-2 wrote:
> >
> > Stephano,
> >
> > THANKS.
> >
> > John
> >
> > On 3/8/07, Stefano Bagnara <ap...@bago.org> wrote:
> >>
> >> You can't use the same db table for 2 different repositories instances
> >> because the locking is at an application level. You would need to
> >> rewrite a distributed locking support (someone did something about this
> >> in past: see james-ha project on sourceforge).
> >>
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Multiple-James-instances-can-use-database-repository--tf3369616.html#a9406721
> Sent from the James - Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
> For additional commands, e-mail: server-user-help@james.apache.org
>
>
Re: [2] Multiple James instances can use database repository?
Posted by sunsetartproducts <ha...@yahoo.com>.
I use this set-up for the whitelist and spam analysis with 2 instances of
James 2.3.0 on different servers sharing a MySql database on a third server.
One machine is the fallover machine the other the primary mail server. I
have not tried using the database to hold the mails, just the whitelist and
spam database. But at least in theory Mysql would allow access by two
different instances, as the file locking in MySQL is at the database level.
When I am using it in this way, I can add or remove data from the database
from either mail server, MySQL just waits to finish the first process before
committing to the next. (not much of a wait as the processes take only a
millisecond or less.) Of course the servers are only 10 feet apart and have
only one hop on a switch to navigate.
If you are using a file system repository for the mails data instead of a
SQL Server database, then you would run into the problem of file locking if
the two servers were trying to access the same files on the hard drive at
the same time.
John G. Norman-2 wrote:
>
> Stephano,
>
> THANKS.
>
> John
>
> On 3/8/07, Stefano Bagnara <ap...@bago.org> wrote:
>>
>> You can't use the same db table for 2 different repositories instances
>> because the locking is at an application level. You would need to
>> rewrite a distributed locking support (someone did something about this
>> in past: see james-ha project on sourceforge).
>>
>
>
--
View this message in context: http://www.nabble.com/Multiple-James-instances-can-use-database-repository--tf3369616.html#a9406721
Sent from the James - Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: Multiple James instances can use database repository?
Posted by "John G. Norman" <jo...@7fff.com>.
Stephano,
THANKS.
John
On 3/8/07, Stefano Bagnara <ap...@bago.org> wrote:
>
> You can't use the same db table for 2 different repositories instances
> because the locking is at an application level. You would need to
> rewrite a distributed locking support (someone did something about this
> in past: see james-ha project on sourceforge).
>
> You can instead simply use the MX solution:
> the primary will be the only real server including inboxes.
> the secondary will simply accept mail destinated to the primary domain,
> but it will then keep it in its own remotedelivery to be delivered to
> the primary as soon as it will be alive again.
>
> Stefano
>
> John G. Norman ha scritto:
> > Hi.
> >
> > I'm thinking a bit about failover and James.
> >
> > If I'm running two Jameses simultaneously, one on each of two systems,
> can
> > they point to the same database-backed repositories (
> > http://james.apache.org/server/2.3.0/repositories.html)?
> >
> > One would be primary MX, the other secondary. So typically, one would
> > probably be handling most of the load, though doubtless some senders
> would
> > hit the secondary for the usual variety of reasons.
> >
> > The load is significant, but not massive; I only use James for bounce
> > control (everything else on Postfix).
> >
> > I saw some ancient discussion such as this:
> >
> > http://marc.theaimsgroup.com/?l=james-user&m=100434648513598&w=2
> >
> > John
> >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
> For additional commands, e-mail: server-user-help@james.apache.org
>
>
Re: Multiple James instances can use database repository?
Posted by Stefano Bagnara <ap...@bago.org>.
You can't use the same db table for 2 different repositories instances
because the locking is at an application level. You would need to
rewrite a distributed locking support (someone did something about this
in past: see james-ha project on sourceforge).
You can instead simply use the MX solution:
the primary will be the only real server including inboxes.
the secondary will simply accept mail destinated to the primary domain,
but it will then keep it in its own remotedelivery to be delivered to
the primary as soon as it will be alive again.
Stefano
John G. Norman ha scritto:
> Hi.
>
> I'm thinking a bit about failover and James.
>
> If I'm running two Jameses simultaneously, one on each of two systems, can
> they point to the same database-backed repositories (
> http://james.apache.org/server/2.3.0/repositories.html)?
>
> One would be primary MX, the other secondary. So typically, one would
> probably be handling most of the load, though doubtless some senders would
> hit the secondary for the usual variety of reasons.
>
> The load is significant, but not massive; I only use James for bounce
> control (everything else on Postfix).
>
> I saw some ancient discussion such as this:
>
> http://marc.theaimsgroup.com/?l=james-user&m=100434648513598&w=2
>
> John
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org