You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-user@db.apache.org by ad marginem <ad...@gmail.com> on 2007/09/18 10:29:31 UTC
Derby and replication
Hello,
I want to develop a program, using derby as a back-end.
There will be a POS and a Warehouse module. They will use the same database
scheme, but they cannot use the same database because of reliability (in
case of broken connection the POS must not stop).
So my question is: how to develop such a reliable workflow? Replication? Or
maybe there are some other options? If replication - how to implement it?
Are there any existing methods to replicate derby database?
Thank you!
--
по краям
Re: Derby and replication
Posted by Kurt Huwig <k....@iku-ag.de>.
Hi,
I am using HA-JDBC to replicate Derby or MySQL:
http://ha-jdbc.sf.net/
It is a JDBC-driver which accesses other JDBC-drivers to execute all
modification statements on all nodes. Reading can be done on one node
or distributed.
Am Dienstag, 18. September 2007 schrieb ad marginem:
> Hello,
>
> I want to develop a program, using derby as a back-end.
>
> There will be a POS and a Warehouse module. They will use the same
> database scheme, but they cannot use the same database because of
> reliability (in case of broken connection the POS must not stop).
>
> So my question is: how to develop such a reliable workflow?
> Replication? Or maybe there are some other options? If replication -
> how to implement it? Are there any existing methods to replicate
> derby database?
>
> Thank you!
--
Mit freundlichen Grüßen
Kurt Huwig (Vorstand)
Telefon 0681/96751-50, Telefax 0681/96751-66
http://www.iku-ag.de/
iKu Systemhaus AG, Am Römerkastell 4, 66121 Saarbrücken
Amtsgericht: Saarbrücken, HRB 13240
Vorstand: Kurt Huwig, Andreas Niederländer
Aufsichtsratsvorsitzender: Jan Bankstahl
GnuPG 1024D/99DD9468 64B1 0C5B 82BC E16E 8940 EB6D 4C32 F908 99DD 9468
Re: Derby and replication
Posted by Jørgen Løland <Jo...@Sun.COM>.
msegel@dbrack01.segel.com wrote:
> You can have asynch replication without having any data loss. Or rather the probability of data loss is minimal.
> Sent via BlackBerry by AT&T
You are right, but you have to calculate with that risk. For database
systems, asynchronous replication comes with the risk that the
transactions committed the last x (milli)seconds before the master
crashes will not be reflected at the slave. That is, there *may* be loss
of information, but does not have to.
The reason is that in contrast to synchronous replication strategies, a
transaction's updates are not force-sent to the slave when it commits.
If the master is running for two years before crashing, and the slave
lags 1 second behind, the data loss is minimal. Depending on your
application, this may or may not be acceptable. For example, I doubt
that even this small % of data loss will be acceptable for money
transfers between banks. It may still be good enough for an email server...
I'd say that you are looking for...
... replication if you want one master DBMS to own the data [1], but
with backup DBMSs ready to take over the responsibilities of the master
role if the current master crashes. The clients will have to reconnect
to a new node [2], but are otherwise not affected by the crash.
... synchronization if you don't want a backup system to be ready to
become a master after a crash. What you need is a central database that
you connect to every now and then to hear the latest news and possibly
append updates (with the risk of dependency conflicts because the client
did not know about updates made by another client).
[1] Systems where the data are distributed across multiple nodes are not
taken into account.
[2] A middle tier, e.g. the JDBC driver, may mask this reconnect for the
clients.
*Regarding the question from the thread starter:* Will it be ok for POS
to have data that are not completely fresh? If an hour or so lag is ok,
you may find what you are looking for in Derby's "Online Backup"
feature. However, this solution will *not* allow updates of the POS
database (nor will the replication feature I mentioned earlier).
--
Jørgen Løland
Re: Derby and replication
Posted by ms...@dbrack01.segel.com.
You can have asynch replication without having any data loss. Or rather the probability of data loss is minimal.
Sent via BlackBerry by AT&T
-----Original Message-----
From: "David Van Couvering" <da...@vancouvering.com>
Date: Tue, 18 Sep 2007 10:39:49
To:"Derby Discussion" <de...@db.apache.org>
Subject: Re: Derby and replication
I don't think it matters so much what the architecture is (e.g.
peer-to-peer versus client-server or whatever). I think what matters
is the use case.
I generally use the word "replication" to imply keeping two instances
tightly in sync for high availability. You have asynchronous
replication, where there is a time lag and a potential loss of data
(this is often used to work across WANs for disaster recover
scenarios) and synchronous replication (this is more common, and is
used to provide a hot standby in case of hardware or software
failure).
I generally use the word "synchronization" to represent the use case
where you want to keep two databases "in synch" for performance
reasons, or because the application using the data may not be able to
connect back to the "main" server. Common use cases are
* keeping a local database on a mobile client like a laptop or PDA
that is a copy of a remote database (think Palm Sync, calendaring apps
on your phone, sales force automation, a local copy of a sales
catalog, etc.). The POS system discussed here is this kind of model -
you keep a local database on the POS terminal and synch it up with the
central POS database
* Having a local database embedded or on the same machine as a web
server or other "edge" server, and this is kept in synch with the Big
Database in the back room. This is attractive as it boosts
performance without giving up transactional consistency. This is
basically a "relational cache."
David
On 9/18/07, Eric Fesh <fe...@verarisoft.com> wrote:
> David:
>
> I've been looking into replication technologies for improvements on a project that I'm working on... The difference between
> replication and synchronization is something I hadn't realized before. Would synchronization be more appropriate for a
> peer-to-peer architecture than replication would be?
>
>
> David Van Couvering wrote:
> > I think the responses you have heard so far talk about replication to
> > provide availability (HA-JDBC, the ongoing Derby effort).
> >
> > What you are talking about is basically using Derby as a reliable
> > front-end cache to a back-end database where the connection can be
> > unavailable from time to time.
> >
> > What this is about is really synchronization, not replication.
> > Synchronization is keeping two sometimes disconnected databases in
> > synch.
> >
> > One product out there that works with Derby is Daffodil Replicator,
> > see http://sourceforge.net/projects/daffodilreplica/. It's an open
> > source project using the GPL license.
> >
> > David
> >
> > On 9/18/07, ad marginem <ad...@gmail.com> wrote:
> >> Hello,
> >>
> >> I want to develop a program, using derby as a back-end.
> >>
> >> There will be a POS and a Warehouse module. They will use the same database
> >> scheme, but they cannot use the same database because of reliability (in
> >> case of broken connection the POS must not stop).
> >>
> >> So my question is: how to develop such a reliable workflow? Replication? Or
> >> maybe there are some other options? If replication - how to implement it?
> >> Are there any existing methods to replicate derby database?
> >>
> >> Thank you!
> >>
> >>
> >> --
> >> по краям
>
> --
> --
>
> Eric Fesh
> Software Engineer
> Verari Systems Software, Inc.
> 110 12th Street North, Suite D103
> Birmingham, AL 35203
> Phone +1-205-397-3141 ext. 3151
> Fax +1-205-397-3142
> fesh@verarisoft.com
> http://www.verarisoft.com/
>
> The information contained in this communication may be confidential and is
> intended only for the use of the recipient(s) named above. If the reader of
> this communication is not the intended recipient(s), you are hereby notified
> that any dissemination, distribution, or copying of this communication, or
> any of its contents, is strictly prohibited. If you are not a named
> recipient or received this communication by mistake, please notify the sender
> and delete the communication and all copies of it.
>
> --
>
> The information contained in this communication may be confidential and is
> intended only for the use of the recipient(s) named above. If the reader
> of this communication is not the intended recipient(s), you are hereby
> notified that any dissemination, distribution, or copying of this
> communication, or any of its contents, is strictly prohibited. If you are
> not a named recipient or received this communication by mistake, please
> notify the sender and delete the communication and all copies of it.
>
>
>
Re: Derby and replication
Posted by David Van Couvering <da...@vancouvering.com>.
I don't think it matters so much what the architecture is (e.g.
peer-to-peer versus client-server or whatever). I think what matters
is the use case.
I generally use the word "replication" to imply keeping two instances
tightly in sync for high availability. You have asynchronous
replication, where there is a time lag and a potential loss of data
(this is often used to work across WANs for disaster recover
scenarios) and synchronous replication (this is more common, and is
used to provide a hot standby in case of hardware or software
failure).
I generally use the word "synchronization" to represent the use case
where you want to keep two databases "in synch" for performance
reasons, or because the application using the data may not be able to
connect back to the "main" server. Common use cases are
* keeping a local database on a mobile client like a laptop or PDA
that is a copy of a remote database (think Palm Sync, calendaring apps
on your phone, sales force automation, a local copy of a sales
catalog, etc.). The POS system discussed here is this kind of model -
you keep a local database on the POS terminal and synch it up with the
central POS database
* Having a local database embedded or on the same machine as a web
server or other "edge" server, and this is kept in synch with the Big
Database in the back room. This is attractive as it boosts
performance without giving up transactional consistency. This is
basically a "relational cache."
David
On 9/18/07, Eric Fesh <fe...@verarisoft.com> wrote:
> David:
>
> I've been looking into replication technologies for improvements on a project that I'm working on... The difference between
> replication and synchronization is something I hadn't realized before. Would synchronization be more appropriate for a
> peer-to-peer architecture than replication would be?
>
>
> David Van Couvering wrote:
> > I think the responses you have heard so far talk about replication to
> > provide availability (HA-JDBC, the ongoing Derby effort).
> >
> > What you are talking about is basically using Derby as a reliable
> > front-end cache to a back-end database where the connection can be
> > unavailable from time to time.
> >
> > What this is about is really synchronization, not replication.
> > Synchronization is keeping two sometimes disconnected databases in
> > synch.
> >
> > One product out there that works with Derby is Daffodil Replicator,
> > see http://sourceforge.net/projects/daffodilreplica/. It's an open
> > source project using the GPL license.
> >
> > David
> >
> > On 9/18/07, ad marginem <ad...@gmail.com> wrote:
> >> Hello,
> >>
> >> I want to develop a program, using derby as a back-end.
> >>
> >> There will be a POS and a Warehouse module. They will use the same database
> >> scheme, but they cannot use the same database because of reliability (in
> >> case of broken connection the POS must not stop).
> >>
> >> So my question is: how to develop such a reliable workflow? Replication? Or
> >> maybe there are some other options? If replication - how to implement it?
> >> Are there any existing methods to replicate derby database?
> >>
> >> Thank you!
> >>
> >>
> >> --
> >> по краям
>
> --
> --
>
> Eric Fesh
> Software Engineer
> Verari Systems Software, Inc.
> 110 12th Street North, Suite D103
> Birmingham, AL 35203
> Phone +1-205-397-3141 ext. 3151
> Fax +1-205-397-3142
> fesh@verarisoft.com
> http://www.verarisoft.com/
>
> The information contained in this communication may be confidential and is
> intended only for the use of the recipient(s) named above. If the reader of
> this communication is not the intended recipient(s), you are hereby notified
> that any dissemination, distribution, or copying of this communication, or
> any of its contents, is strictly prohibited. If you are not a named
> recipient or received this communication by mistake, please notify the sender
> and delete the communication and all copies of it.
>
> --
>
> The information contained in this communication may be confidential and is
> intended only for the use of the recipient(s) named above. If the reader
> of this communication is not the intended recipient(s), you are hereby
> notified that any dissemination, distribution, or copying of this
> communication, or any of its contents, is strictly prohibited. If you are
> not a named recipient or received this communication by mistake, please
> notify the sender and delete the communication and all copies of it.
>
>
>
Re: Derby and replication
Posted by Eric Fesh <fe...@verarisoft.com>.
David:
I've been looking into replication technologies for improvements on a project that I'm working on... The difference between
replication and synchronization is something I hadn't realized before. Would synchronization be more appropriate for a
peer-to-peer architecture than replication would be?
David Van Couvering wrote:
> I think the responses you have heard so far talk about replication to
> provide availability (HA-JDBC, the ongoing Derby effort).
>
> What you are talking about is basically using Derby as a reliable
> front-end cache to a back-end database where the connection can be
> unavailable from time to time.
>
> What this is about is really synchronization, not replication.
> Synchronization is keeping two sometimes disconnected databases in
> synch.
>
> One product out there that works with Derby is Daffodil Replicator,
> see http://sourceforge.net/projects/daffodilreplica/. It's an open
> source project using the GPL license.
>
> David
>
> On 9/18/07, ad marginem <ad...@gmail.com> wrote:
>> Hello,
>>
>> I want to develop a program, using derby as a back-end.
>>
>> There will be a POS and a Warehouse module. They will use the same database
>> scheme, but they cannot use the same database because of reliability (in
>> case of broken connection the POS must not stop).
>>
>> So my question is: how to develop such a reliable workflow? Replication? Or
>> maybe there are some other options? If replication - how to implement it?
>> Are there any existing methods to replicate derby database?
>>
>> Thank you!
>>
>>
>> --
>> по краям
--
--
Eric Fesh
Software Engineer
Verari Systems Software, Inc.
110 12th Street North, Suite D103
Birmingham, AL 35203
Phone +1-205-397-3141 ext. 3151
Fax +1-205-397-3142
fesh@verarisoft.com
http://www.verarisoft.com/
The information contained in this communication may be confidential and is
intended only for the use of the recipient(s) named above. If the reader of
this communication is not the intended recipient(s), you are hereby notified
that any dissemination, distribution, or copying of this communication, or
any of its contents, is strictly prohibited. If you are not a named
recipient or received this communication by mistake, please notify the sender
and delete the communication and all copies of it.
--
The information contained in this communication may be confidential and is
intended only for the use of the recipient(s) named above. If the reader
of this communication is not the intended recipient(s), you are hereby
notified that any dissemination, distribution, or copying of this
communication, or any of its contents, is strictly prohibited. If you are
not a named recipient or received this communication by mistake, please
notify the sender and delete the communication and all copies of it.
Re: Derby and replication
Posted by David Van Couvering <da...@vancouvering.com>.
I think you should take a look at Daffodil. I think it works across
machines. I've never used it, but others who have, I thought, used it
across machines.
David
On 9/19/07, John C. Turnbull <oz...@ozemail.com.au> wrote:
> I am investigating the ins and outs of a system that consists of several
> JVMs running on possibly separate machines all manipulating separate
> instances of Derby that collectively are kept in synch - somehow. Does
> anyone know of a possible solution to the synchronisation side of things or
> am I dreaming? I am not sure if HA-JDBC or Daffodil Replicator can handle
> the inter-machine communication issues.
>
> -JCT
>
> > -----Original Message-----
> > From: david.vancouvering@gmail.com
> > [mailto:david.vancouvering@gmail.com] On Behalf Of David Van Couvering
> > Sent: Wednesday, 19 September 2007 03:17
> > To: Derby Discussion
> > Subject: Re: Derby and replication
> >
> > I think the responses you have heard so far talk about replication to
> > provide availability (HA-JDBC, the ongoing Derby effort).
> >
> > What you are talking about is basically using Derby as a reliable
> > front-end cache to a back-end database where the connection can be
> > unavailable from time to time.
> >
> > What this is about is really synchronization, not replication.
> > Synchronization is keeping two sometimes disconnected databases in
> > synch.
> >
> > One product out there that works with Derby is Daffodil Replicator,
> > see http://sourceforge.net/projects/daffodilreplica/. It's an open
> > source project using the GPL license.
> >
> > David
> >
> > On 9/18/07, ad marginem <ad...@gmail.com> wrote:
> > > Hello,
> > >
> > > I want to develop a program, using derby as a back-end.
> > >
> > > There will be a POS and a Warehouse module. They will use the same
> > database
> > > scheme, but they cannot use the same database because of reliability
> > (in
> > > case of broken connection the POS must not stop).
> > >
> > > So my question is: how to develop such a reliable workflow?
> > Replication? Or
> > > maybe there are some other options? If replication - how to implement
> > it?
> > > Are there any existing methods to replicate derby database?
> > >
> > > Thank you!
> > >
> > >
> > > --
> > > по краям
>
RE: Derby and replication
Posted by "John C. Turnbull" <oz...@ozemail.com.au>.
I am investigating the ins and outs of a system that consists of several
JVMs running on possibly separate machines all manipulating separate
instances of Derby that collectively are kept in synch - somehow. Does
anyone know of a possible solution to the synchronisation side of things or
am I dreaming? I am not sure if HA-JDBC or Daffodil Replicator can handle
the inter-machine communication issues.
-JCT
> -----Original Message-----
> From: david.vancouvering@gmail.com
> [mailto:david.vancouvering@gmail.com] On Behalf Of David Van Couvering
> Sent: Wednesday, 19 September 2007 03:17
> To: Derby Discussion
> Subject: Re: Derby and replication
>
> I think the responses you have heard so far talk about replication to
> provide availability (HA-JDBC, the ongoing Derby effort).
>
> What you are talking about is basically using Derby as a reliable
> front-end cache to a back-end database where the connection can be
> unavailable from time to time.
>
> What this is about is really synchronization, not replication.
> Synchronization is keeping two sometimes disconnected databases in
> synch.
>
> One product out there that works with Derby is Daffodil Replicator,
> see http://sourceforge.net/projects/daffodilreplica/. It's an open
> source project using the GPL license.
>
> David
>
> On 9/18/07, ad marginem <ad...@gmail.com> wrote:
> > Hello,
> >
> > I want to develop a program, using derby as a back-end.
> >
> > There will be a POS and a Warehouse module. They will use the same
> database
> > scheme, but they cannot use the same database because of reliability
> (in
> > case of broken connection the POS must not stop).
> >
> > So my question is: how to develop such a reliable workflow?
> Replication? Or
> > maybe there are some other options? If replication - how to implement
> it?
> > Are there any existing methods to replicate derby database?
> >
> > Thank you!
> >
> >
> > --
> > по краям
Re: Derby and replication
Posted by David Van Couvering <da...@vancouvering.com>.
I think the responses you have heard so far talk about replication to
provide availability (HA-JDBC, the ongoing Derby effort).
What you are talking about is basically using Derby as a reliable
front-end cache to a back-end database where the connection can be
unavailable from time to time.
What this is about is really synchronization, not replication.
Synchronization is keeping two sometimes disconnected databases in
synch.
One product out there that works with Derby is Daffodil Replicator,
see http://sourceforge.net/projects/daffodilreplica/. It's an open
source project using the GPL license.
David
On 9/18/07, ad marginem <ad...@gmail.com> wrote:
> Hello,
>
> I want to develop a program, using derby as a back-end.
>
> There will be a POS and a Warehouse module. They will use the same database
> scheme, but they cannot use the same database because of reliability (in
> case of broken connection the POS must not stop).
>
> So my question is: how to develop such a reliable workflow? Replication? Or
> maybe there are some other options? If replication - how to implement it?
> Are there any existing methods to replicate derby database?
>
> Thank you!
>
>
> --
> по краям
Re: Derby and replication
Posted by Jørgen Løland <Jo...@Sun.COM>.
Hi Ad Marginem
There is an ongoing effort to add replication to Derby that we hope will
be ready for the next major Derby release. The currently planned
functionality will add replication for availability improvement only,
and reading/updating the "back-end" database will not be allowed.
However, if more developers took part in this issue, the scope could be
widened to, e.g., allow "back-end" reads.
A "new feature" request has already been filed for replication in
Derby's bug tracker here:
https://issues.apache.org/jira/browse/DERBY-2872. You are more than
welcome to participate in the development of this (and other) Derby
issues. As a starting point, you could have a look at the functional
specification posted on the issue mentioned above and post any comments
you might have.
Jørgen Løland
ad marginem wrote:
> Hello,
>
> I want to develop a program, using derby as a back-end.
>
> There will be a POS and a Warehouse module. They will use the same database
> scheme, but they cannot use the same database because of reliability (in
> case of broken connection the POS must not stop).
>
> So my question is: how to develop such a reliable workflow? Replication? Or
> maybe there are some other options? If replication - how to implement it?
> Are there any existing methods to replicate derby database?
>
> Thank you!
>
>