You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Egil Sørensen (JIRA)" <ji...@apache.org> on 2007/06/21 12:05:25 UTC
[jira] Created: (DERBY-2852) Derby: Replication and Availability
Derby: Replication and Availability
-----------------------------------
Key: DERBY-2852
URL: https://issues.apache.org/jira/browse/DERBY-2852
Project: Derby
Issue Type: New Feature
Components: Network Server, Services, SQL, Store
Affects Versions: 10.2.2.0
Reporter: Egil Sørensen
This paper and attached source code describes the work done to add hot standby replication functionality to the Apache Derby Database Management System.
By implementing a hot standby scheme in Apache Derby several features are added. The contents of the database is replicated at run time to another site providing online runtime backup. As the hot standby takes over on faults availability is added in that a client can connect to the hot standby after a crash. Thus the crash is masked from the clients. In addition to this, online upgrades of software and hardware can be done by taking down one database at the time. Then when
the upgrade is completed the upgraded server is synchronized and back online with no downtime.
A fully functional prototype of the Apache Derby hot standby scheme has been created in this project using logical logs, fail-fast takeovers and logical catchups after an internal up-to-crash recovery and reconnection. This project builds on the ideas that are presented in my earlier project: Derby: Write to Neighbor Mode.
To enable the features described above some changes have been made to the Derby base system. The following classes have been changed or added:
The Logical Log System
org.apache.derby.iapi.store.logical.log.LogicalImage
org.apache.derby.iapi.store.logical.log.LogicalLoggable
org.apache.derby.impl.store.logical.log.LogicalLog
org.apache.derby.impl.store.logical.log.LogicalLogger
org.apache.derby.impl.store.logical.log.LogicalLogRecord
org.apache.derby.impl.store.logical.opearations.LogicalAbortOperation
org.apache.derby.impl.store.logical.opearations.LogicalCommitOperation
org.apache.derby.impl.store.logical.opearations.LogicalDeleteOperation
org.apache.derby.impl.store.logical.opearations.LogicalImage
org.apache.derby.impl.store.logical.opearations.LogicalInsertOperation
org.apache.derby.impl.store.logical.opearations.LogicalOperation
org.apache.derby.impl.store.logical.opearations.LogicalOperationFactory
org.apache.derby.impl.store.logical.opearations.LogicalUpdateOperation
The SQL Layer
org.apache.impl.sql.execute.RowChangerImpl
org.apache.impl.sql.conn.GenericLanguageConnectionContext
The Communications Service
org.apache.derby.impl.services.net.LogicalServer
org.apache.derby.impl.services.net.LogicalClient
org.apache.derby.impl.services.net.NetworkPayload
org.apache.derby.impl.services.net.receiver.LogReceiver
org.apache.derby.impl.services.net.shipper.LogShipper
org.apache.derby.impl.services.net.shipper.LogicalCatchUp
The Hot Standby Service
org.apache.derby.impl.services.hotstandby.HotStandbyController
org.apache.derby.impl.services.hotstandby.Database
The Network Server
org.apache.derby.impl.drda.NetworkServerControlImpl
org.apache.derby.impl.drda.HotStandbyPoller
org.apache.derby.impl.drda.net.LogicalClient
org.apache.derby.imp.drda.net.LogicalServer
See the added paper for more thorough documentation on design choices and implementation along with some benchmarking results
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2852) Derby: Replication and Availability
Posted by "Egil Sørensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12507845 ]
Egil Sørensen commented on DERBY-2852:
--------------------------------------
Ah, the java.io.FileNotFoundException: logicallog/logical.log (No such file or directory) is of course due to the folder 'logicallog' missing from the path the database is run from. To fix this, either create a folder named logicallog or replace the path in org.apache.derby.impl.store.logical.logicallog#flushToDisk
I'm sorry for not informing you about this. Thanks!
> Derby: Replication and Availability
> -----------------------------------
>
> Key: DERBY-2852
> URL: https://issues.apache.org/jira/browse/DERBY-2852
> Project: Derby
> Issue Type: New Feature
> Components: Network Server, Services, SQL, Store
> Affects Versions: 10.2.2.0
> Reporter: Egil Sørensen
> Attachments: derby-rep.zip, repl_trunk.diff, repl_trunk.stat, repl_trunk_ignored.diff, soereeg07, svn.diff
>
>
> This paper and attached source code describes the work done to add hot standby replication functionality to the Apache Derby Database Management System.
> By implementing a hot standby scheme in Apache Derby several features are added. The contents of the database is replicated at run time to another site providing online runtime backup. As the hot standby takes over on faults availability is added in that a client can connect to the hot standby after a crash. Thus the crash is masked from the clients. In addition to this, online upgrades of software and hardware can be done by taking down one database at the time. Then when
> the upgrade is completed the upgraded server is synchronized and back online with no downtime.
> A fully functional prototype of the Apache Derby hot standby scheme has been created in this project using logical logs, fail-fast takeovers and logical catchups after an internal up-to-crash recovery and reconnection. This project builds on the ideas that are presented in my earlier project: Derby: Write to Neighbor Mode.
> To enable the features described above some changes have been made to the Derby base system. The following classes have been changed or added:
> The Logical Log System
> org.apache.derby.iapi.store.logical.log.LogicalImage
> org.apache.derby.iapi.store.logical.log.LogicalLoggable
> org.apache.derby.impl.store.logical.log.LogicalLog
> org.apache.derby.impl.store.logical.log.LogicalLogger
> org.apache.derby.impl.store.logical.log.LogicalLogRecord
> org.apache.derby.impl.store.logical.opearations.LogicalAbortOperation
> org.apache.derby.impl.store.logical.opearations.LogicalCommitOperation
> org.apache.derby.impl.store.logical.opearations.LogicalDeleteOperation
> org.apache.derby.impl.store.logical.opearations.LogicalImage
> org.apache.derby.impl.store.logical.opearations.LogicalInsertOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperationFactory
> org.apache.derby.impl.store.logical.opearations.LogicalUpdateOperation
> The SQL Layer
> org.apache.impl.sql.execute.RowChangerImpl
> org.apache.impl.sql.conn.GenericLanguageConnectionContext
> The Communications Service
> org.apache.derby.impl.services.net.LogicalServer
> org.apache.derby.impl.services.net.LogicalClient
> org.apache.derby.impl.services.net.NetworkPayload
> org.apache.derby.impl.services.net.receiver.LogReceiver
> org.apache.derby.impl.services.net.shipper.LogShipper
> org.apache.derby.impl.services.net.shipper.LogicalCatchUp
> The Hot Standby Service
> org.apache.derby.impl.services.hotstandby.HotStandbyController
> org.apache.derby.impl.services.hotstandby.Database
> The Network Server
> org.apache.derby.impl.drda.NetworkServerControlImpl
> org.apache.derby.impl.drda.HotStandbyPoller
> org.apache.derby.impl.drda.net.LogicalClient
> org.apache.derby.imp.drda.net.LogicalServer
> See the added paper for more thorough documentation on design choices and implementation along with some benchmarking results
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2852) Derby: Replication and Availability
Posted by "Egil Sørensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12517844 ]
Egil Sørensen commented on DERBY-2852:
--------------------------------------
I am terribly sorry that I have not yet submitted the ICLA, I have signed it already, but had to wait for a final clearance from the University that the source code was mine to donate. I will send it tomorrow, you should have it within the week.
Sorry for the inconvenience.
> Derby: Replication and Availability
> -----------------------------------
>
> Key: DERBY-2852
> URL: https://issues.apache.org/jira/browse/DERBY-2852
> Project: Derby
> Issue Type: New Feature
> Components: Network Server, Services, SQL, Store
> Affects Versions: 10.2.2.0
> Reporter: Egil Sørensen
> Attachments: derby-rep.zip, repl_trunk.diff, repl_trunk.stat, repl_trunk_ignored.diff, soereeg07, svn.diff
>
>
> This paper and attached source code describes the work done to add hot standby replication functionality to the Apache Derby Database Management System.
> By implementing a hot standby scheme in Apache Derby several features are added. The contents of the database is replicated at run time to another site providing online runtime backup. As the hot standby takes over on faults availability is added in that a client can connect to the hot standby after a crash. Thus the crash is masked from the clients. In addition to this, online upgrades of software and hardware can be done by taking down one database at the time. Then when
> the upgrade is completed the upgraded server is synchronized and back online with no downtime.
> A fully functional prototype of the Apache Derby hot standby scheme has been created in this project using logical logs, fail-fast takeovers and logical catchups after an internal up-to-crash recovery and reconnection. This project builds on the ideas that are presented in my earlier project: Derby: Write to Neighbor Mode.
> To enable the features described above some changes have been made to the Derby base system. The following classes have been changed or added:
> The Logical Log System
> org.apache.derby.iapi.store.logical.log.LogicalImage
> org.apache.derby.iapi.store.logical.log.LogicalLoggable
> org.apache.derby.impl.store.logical.log.LogicalLog
> org.apache.derby.impl.store.logical.log.LogicalLogger
> org.apache.derby.impl.store.logical.log.LogicalLogRecord
> org.apache.derby.impl.store.logical.opearations.LogicalAbortOperation
> org.apache.derby.impl.store.logical.opearations.LogicalCommitOperation
> org.apache.derby.impl.store.logical.opearations.LogicalDeleteOperation
> org.apache.derby.impl.store.logical.opearations.LogicalImage
> org.apache.derby.impl.store.logical.opearations.LogicalInsertOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperationFactory
> org.apache.derby.impl.store.logical.opearations.LogicalUpdateOperation
> The SQL Layer
> org.apache.impl.sql.execute.RowChangerImpl
> org.apache.impl.sql.conn.GenericLanguageConnectionContext
> The Communications Service
> org.apache.derby.impl.services.net.LogicalServer
> org.apache.derby.impl.services.net.LogicalClient
> org.apache.derby.impl.services.net.NetworkPayload
> org.apache.derby.impl.services.net.receiver.LogReceiver
> org.apache.derby.impl.services.net.shipper.LogShipper
> org.apache.derby.impl.services.net.shipper.LogicalCatchUp
> The Hot Standby Service
> org.apache.derby.impl.services.hotstandby.HotStandbyController
> org.apache.derby.impl.services.hotstandby.Database
> The Network Server
> org.apache.derby.impl.drda.NetworkServerControlImpl
> org.apache.derby.impl.drda.HotStandbyPoller
> org.apache.derby.impl.drda.net.LogicalClient
> org.apache.derby.imp.drda.net.LogicalServer
> See the added paper for more thorough documentation on design choices and implementation along with some benchmarking results
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2852) Derby: Replication and Availability
Posted by "Egil Sørensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Egil Sørensen updated DERBY-2852:
---------------------------------
Attachment: soereeg07
Documentation on the logical replication scheme added to Apache Derby
> Derby: Replication and Availability
> -----------------------------------
>
> Key: DERBY-2852
> URL: https://issues.apache.org/jira/browse/DERBY-2852
> Project: Derby
> Issue Type: New Feature
> Components: Network Server, Services, SQL, Store
> Affects Versions: 10.2.2.0
> Reporter: Egil Sørensen
> Attachments: soereeg07
>
>
> This paper and attached source code describes the work done to add hot standby replication functionality to the Apache Derby Database Management System.
> By implementing a hot standby scheme in Apache Derby several features are added. The contents of the database is replicated at run time to another site providing online runtime backup. As the hot standby takes over on faults availability is added in that a client can connect to the hot standby after a crash. Thus the crash is masked from the clients. In addition to this, online upgrades of software and hardware can be done by taking down one database at the time. Then when
> the upgrade is completed the upgraded server is synchronized and back online with no downtime.
> A fully functional prototype of the Apache Derby hot standby scheme has been created in this project using logical logs, fail-fast takeovers and logical catchups after an internal up-to-crash recovery and reconnection. This project builds on the ideas that are presented in my earlier project: Derby: Write to Neighbor Mode.
> To enable the features described above some changes have been made to the Derby base system. The following classes have been changed or added:
> The Logical Log System
> org.apache.derby.iapi.store.logical.log.LogicalImage
> org.apache.derby.iapi.store.logical.log.LogicalLoggable
> org.apache.derby.impl.store.logical.log.LogicalLog
> org.apache.derby.impl.store.logical.log.LogicalLogger
> org.apache.derby.impl.store.logical.log.LogicalLogRecord
> org.apache.derby.impl.store.logical.opearations.LogicalAbortOperation
> org.apache.derby.impl.store.logical.opearations.LogicalCommitOperation
> org.apache.derby.impl.store.logical.opearations.LogicalDeleteOperation
> org.apache.derby.impl.store.logical.opearations.LogicalImage
> org.apache.derby.impl.store.logical.opearations.LogicalInsertOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperationFactory
> org.apache.derby.impl.store.logical.opearations.LogicalUpdateOperation
> The SQL Layer
> org.apache.impl.sql.execute.RowChangerImpl
> org.apache.impl.sql.conn.GenericLanguageConnectionContext
> The Communications Service
> org.apache.derby.impl.services.net.LogicalServer
> org.apache.derby.impl.services.net.LogicalClient
> org.apache.derby.impl.services.net.NetworkPayload
> org.apache.derby.impl.services.net.receiver.LogReceiver
> org.apache.derby.impl.services.net.shipper.LogShipper
> org.apache.derby.impl.services.net.shipper.LogicalCatchUp
> The Hot Standby Service
> org.apache.derby.impl.services.hotstandby.HotStandbyController
> org.apache.derby.impl.services.hotstandby.Database
> The Network Server
> org.apache.derby.impl.drda.NetworkServerControlImpl
> org.apache.derby.impl.drda.HotStandbyPoller
> org.apache.derby.impl.drda.net.LogicalClient
> org.apache.derby.imp.drda.net.LogicalServer
> See the added paper for more thorough documentation on design choices and implementation along with some benchmarking results
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2852) Derby: Replication and Availability
Posted by "Jørgen Løland (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jørgen Løland updated DERBY-2852:
---------------------------------
Derby Info: (was: [Patch Available])
Unchecking Patch Available.
The code is an excellent proof of concept for the functionality (after all, it works :) ), but after studying it I do think there is more work that needs to be done for it to be considered for inclusion in Derby.
Egil: feel free to change to "patch available" again if you do not agree but think the patch should be considered for inclusion as it is.
> Derby: Replication and Availability
> -----------------------------------
>
> Key: DERBY-2852
> URL: https://issues.apache.org/jira/browse/DERBY-2852
> Project: Derby
> Issue Type: New Feature
> Components: Network Server, Services, SQL, Store
> Affects Versions: 10.2.2.0
> Reporter: Egil Sørensen
> Attachments: derby-rep.zip, repl_trunk.diff, repl_trunk.stat, repl_trunk_ignored.diff, soereeg07, svn.diff
>
>
> This paper and attached source code describes the work done to add hot standby replication functionality to the Apache Derby Database Management System.
> By implementing a hot standby scheme in Apache Derby several features are added. The contents of the database is replicated at run time to another site providing online runtime backup. As the hot standby takes over on faults availability is added in that a client can connect to the hot standby after a crash. Thus the crash is masked from the clients. In addition to this, online upgrades of software and hardware can be done by taking down one database at the time. Then when
> the upgrade is completed the upgraded server is synchronized and back online with no downtime.
> A fully functional prototype of the Apache Derby hot standby scheme has been created in this project using logical logs, fail-fast takeovers and logical catchups after an internal up-to-crash recovery and reconnection. This project builds on the ideas that are presented in my earlier project: Derby: Write to Neighbor Mode.
> To enable the features described above some changes have been made to the Derby base system. The following classes have been changed or added:
> The Logical Log System
> org.apache.derby.iapi.store.logical.log.LogicalImage
> org.apache.derby.iapi.store.logical.log.LogicalLoggable
> org.apache.derby.impl.store.logical.log.LogicalLog
> org.apache.derby.impl.store.logical.log.LogicalLogger
> org.apache.derby.impl.store.logical.log.LogicalLogRecord
> org.apache.derby.impl.store.logical.opearations.LogicalAbortOperation
> org.apache.derby.impl.store.logical.opearations.LogicalCommitOperation
> org.apache.derby.impl.store.logical.opearations.LogicalDeleteOperation
> org.apache.derby.impl.store.logical.opearations.LogicalImage
> org.apache.derby.impl.store.logical.opearations.LogicalInsertOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperationFactory
> org.apache.derby.impl.store.logical.opearations.LogicalUpdateOperation
> The SQL Layer
> org.apache.impl.sql.execute.RowChangerImpl
> org.apache.impl.sql.conn.GenericLanguageConnectionContext
> The Communications Service
> org.apache.derby.impl.services.net.LogicalServer
> org.apache.derby.impl.services.net.LogicalClient
> org.apache.derby.impl.services.net.NetworkPayload
> org.apache.derby.impl.services.net.receiver.LogReceiver
> org.apache.derby.impl.services.net.shipper.LogShipper
> org.apache.derby.impl.services.net.shipper.LogicalCatchUp
> The Hot Standby Service
> org.apache.derby.impl.services.hotstandby.HotStandbyController
> org.apache.derby.impl.services.hotstandby.Database
> The Network Server
> org.apache.derby.impl.drda.NetworkServerControlImpl
> org.apache.derby.impl.drda.HotStandbyPoller
> org.apache.derby.impl.drda.net.LogicalClient
> org.apache.derby.imp.drda.net.LogicalServer
> See the added paper for more thorough documentation on design choices and implementation along with some benchmarking results
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2852) Derby: Replication and Availability
Posted by "Jørgen Løland (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jørgen Løland updated DERBY-2852:
---------------------------------
Attachment: repl_trunk_ignored.diff
repl_trunk.stat
repl_trunk.diff
Egil,
I think the posted replication functionality looks really interesting. I have had a quick look at your code, and tried to make a patch for the current trunk version to make it easier for other developers to test it.
repl_trunk.* contains the patch, while repl_trunk_ignored.diff contains changes in the originally posted diff that I *believe* should not be part of the replication patch (maybe noise from diff'ing between two different derby revisions?)
Note that the attached files are almost equal to what was posted by Egil in the zip and diff files. I have only made some small changes to make it fit into trunk. Furthermore, the files that were in the zip file (added files) are now found in the diff file.
I have had a few test runs and have encountered a few of these on the Derby running as "primary":
java.io.FileNotFoundException: logicallog/logical.log (No such file or directory)
No debugging has been performed on this.
> Derby: Replication and Availability
> -----------------------------------
>
> Key: DERBY-2852
> URL: https://issues.apache.org/jira/browse/DERBY-2852
> Project: Derby
> Issue Type: New Feature
> Components: Network Server, Services, SQL, Store
> Affects Versions: 10.2.2.0
> Reporter: Egil Sørensen
> Attachments: derby-rep.zip, repl_trunk.diff, repl_trunk.stat, repl_trunk_ignored.diff, soereeg07, svn.diff
>
>
> This paper and attached source code describes the work done to add hot standby replication functionality to the Apache Derby Database Management System.
> By implementing a hot standby scheme in Apache Derby several features are added. The contents of the database is replicated at run time to another site providing online runtime backup. As the hot standby takes over on faults availability is added in that a client can connect to the hot standby after a crash. Thus the crash is masked from the clients. In addition to this, online upgrades of software and hardware can be done by taking down one database at the time. Then when
> the upgrade is completed the upgraded server is synchronized and back online with no downtime.
> A fully functional prototype of the Apache Derby hot standby scheme has been created in this project using logical logs, fail-fast takeovers and logical catchups after an internal up-to-crash recovery and reconnection. This project builds on the ideas that are presented in my earlier project: Derby: Write to Neighbor Mode.
> To enable the features described above some changes have been made to the Derby base system. The following classes have been changed or added:
> The Logical Log System
> org.apache.derby.iapi.store.logical.log.LogicalImage
> org.apache.derby.iapi.store.logical.log.LogicalLoggable
> org.apache.derby.impl.store.logical.log.LogicalLog
> org.apache.derby.impl.store.logical.log.LogicalLogger
> org.apache.derby.impl.store.logical.log.LogicalLogRecord
> org.apache.derby.impl.store.logical.opearations.LogicalAbortOperation
> org.apache.derby.impl.store.logical.opearations.LogicalCommitOperation
> org.apache.derby.impl.store.logical.opearations.LogicalDeleteOperation
> org.apache.derby.impl.store.logical.opearations.LogicalImage
> org.apache.derby.impl.store.logical.opearations.LogicalInsertOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperationFactory
> org.apache.derby.impl.store.logical.opearations.LogicalUpdateOperation
> The SQL Layer
> org.apache.impl.sql.execute.RowChangerImpl
> org.apache.impl.sql.conn.GenericLanguageConnectionContext
> The Communications Service
> org.apache.derby.impl.services.net.LogicalServer
> org.apache.derby.impl.services.net.LogicalClient
> org.apache.derby.impl.services.net.NetworkPayload
> org.apache.derby.impl.services.net.receiver.LogReceiver
> org.apache.derby.impl.services.net.shipper.LogShipper
> org.apache.derby.impl.services.net.shipper.LogicalCatchUp
> The Hot Standby Service
> org.apache.derby.impl.services.hotstandby.HotStandbyController
> org.apache.derby.impl.services.hotstandby.Database
> The Network Server
> org.apache.derby.impl.drda.NetworkServerControlImpl
> org.apache.derby.impl.drda.HotStandbyPoller
> org.apache.derby.impl.drda.net.LogicalClient
> org.apache.derby.imp.drda.net.LogicalServer
> See the added paper for more thorough documentation on design choices and implementation along with some benchmarking results
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2852) Derby: Replication and Availability
Posted by "Egil Sørensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12509134 ]
Egil Sørensen commented on DERBY-2852:
--------------------------------------
I have planned to submit an ICLA, I intended to do so, but due to my vacation it was forgotten. Hopefully it will be submitted within the next 3 or 4 days.
> Derby: Replication and Availability
> -----------------------------------
>
> Key: DERBY-2852
> URL: https://issues.apache.org/jira/browse/DERBY-2852
> Project: Derby
> Issue Type: New Feature
> Components: Network Server, Services, SQL, Store
> Affects Versions: 10.2.2.0
> Reporter: Egil Sørensen
> Attachments: derby-rep.zip, repl_trunk.diff, repl_trunk.stat, repl_trunk_ignored.diff, soereeg07, svn.diff
>
>
> This paper and attached source code describes the work done to add hot standby replication functionality to the Apache Derby Database Management System.
> By implementing a hot standby scheme in Apache Derby several features are added. The contents of the database is replicated at run time to another site providing online runtime backup. As the hot standby takes over on faults availability is added in that a client can connect to the hot standby after a crash. Thus the crash is masked from the clients. In addition to this, online upgrades of software and hardware can be done by taking down one database at the time. Then when
> the upgrade is completed the upgraded server is synchronized and back online with no downtime.
> A fully functional prototype of the Apache Derby hot standby scheme has been created in this project using logical logs, fail-fast takeovers and logical catchups after an internal up-to-crash recovery and reconnection. This project builds on the ideas that are presented in my earlier project: Derby: Write to Neighbor Mode.
> To enable the features described above some changes have been made to the Derby base system. The following classes have been changed or added:
> The Logical Log System
> org.apache.derby.iapi.store.logical.log.LogicalImage
> org.apache.derby.iapi.store.logical.log.LogicalLoggable
> org.apache.derby.impl.store.logical.log.LogicalLog
> org.apache.derby.impl.store.logical.log.LogicalLogger
> org.apache.derby.impl.store.logical.log.LogicalLogRecord
> org.apache.derby.impl.store.logical.opearations.LogicalAbortOperation
> org.apache.derby.impl.store.logical.opearations.LogicalCommitOperation
> org.apache.derby.impl.store.logical.opearations.LogicalDeleteOperation
> org.apache.derby.impl.store.logical.opearations.LogicalImage
> org.apache.derby.impl.store.logical.opearations.LogicalInsertOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperationFactory
> org.apache.derby.impl.store.logical.opearations.LogicalUpdateOperation
> The SQL Layer
> org.apache.impl.sql.execute.RowChangerImpl
> org.apache.impl.sql.conn.GenericLanguageConnectionContext
> The Communications Service
> org.apache.derby.impl.services.net.LogicalServer
> org.apache.derby.impl.services.net.LogicalClient
> org.apache.derby.impl.services.net.NetworkPayload
> org.apache.derby.impl.services.net.receiver.LogReceiver
> org.apache.derby.impl.services.net.shipper.LogShipper
> org.apache.derby.impl.services.net.shipper.LogicalCatchUp
> The Hot Standby Service
> org.apache.derby.impl.services.hotstandby.HotStandbyController
> org.apache.derby.impl.services.hotstandby.Database
> The Network Server
> org.apache.derby.impl.drda.NetworkServerControlImpl
> org.apache.derby.impl.drda.HotStandbyPoller
> org.apache.derby.impl.drda.net.LogicalClient
> org.apache.derby.imp.drda.net.LogicalServer
> See the added paper for more thorough documentation on design choices and implementation along with some benchmarking results
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2852) Derby: Replication and Availability
Posted by "Daniel John Debrunner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12508179 ]
Daniel John Debrunner commented on DERBY-2852:
----------------------------------------------
Egil, are you planning to submit an ICLA to Apache?
If not, there is the significant chance that this patch cannot be accepted into the Derby codeline.
> Derby: Replication and Availability
> -----------------------------------
>
> Key: DERBY-2852
> URL: https://issues.apache.org/jira/browse/DERBY-2852
> Project: Derby
> Issue Type: New Feature
> Components: Network Server, Services, SQL, Store
> Affects Versions: 10.2.2.0
> Reporter: Egil Sørensen
> Attachments: derby-rep.zip, repl_trunk.diff, repl_trunk.stat, repl_trunk_ignored.diff, soereeg07, svn.diff
>
>
> This paper and attached source code describes the work done to add hot standby replication functionality to the Apache Derby Database Management System.
> By implementing a hot standby scheme in Apache Derby several features are added. The contents of the database is replicated at run time to another site providing online runtime backup. As the hot standby takes over on faults availability is added in that a client can connect to the hot standby after a crash. Thus the crash is masked from the clients. In addition to this, online upgrades of software and hardware can be done by taking down one database at the time. Then when
> the upgrade is completed the upgraded server is synchronized and back online with no downtime.
> A fully functional prototype of the Apache Derby hot standby scheme has been created in this project using logical logs, fail-fast takeovers and logical catchups after an internal up-to-crash recovery and reconnection. This project builds on the ideas that are presented in my earlier project: Derby: Write to Neighbor Mode.
> To enable the features described above some changes have been made to the Derby base system. The following classes have been changed or added:
> The Logical Log System
> org.apache.derby.iapi.store.logical.log.LogicalImage
> org.apache.derby.iapi.store.logical.log.LogicalLoggable
> org.apache.derby.impl.store.logical.log.LogicalLog
> org.apache.derby.impl.store.logical.log.LogicalLogger
> org.apache.derby.impl.store.logical.log.LogicalLogRecord
> org.apache.derby.impl.store.logical.opearations.LogicalAbortOperation
> org.apache.derby.impl.store.logical.opearations.LogicalCommitOperation
> org.apache.derby.impl.store.logical.opearations.LogicalDeleteOperation
> org.apache.derby.impl.store.logical.opearations.LogicalImage
> org.apache.derby.impl.store.logical.opearations.LogicalInsertOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperationFactory
> org.apache.derby.impl.store.logical.opearations.LogicalUpdateOperation
> The SQL Layer
> org.apache.impl.sql.execute.RowChangerImpl
> org.apache.impl.sql.conn.GenericLanguageConnectionContext
> The Communications Service
> org.apache.derby.impl.services.net.LogicalServer
> org.apache.derby.impl.services.net.LogicalClient
> org.apache.derby.impl.services.net.NetworkPayload
> org.apache.derby.impl.services.net.receiver.LogReceiver
> org.apache.derby.impl.services.net.shipper.LogShipper
> org.apache.derby.impl.services.net.shipper.LogicalCatchUp
> The Hot Standby Service
> org.apache.derby.impl.services.hotstandby.HotStandbyController
> org.apache.derby.impl.services.hotstandby.Database
> The Network Server
> org.apache.derby.impl.drda.NetworkServerControlImpl
> org.apache.derby.impl.drda.HotStandbyPoller
> org.apache.derby.impl.drda.net.LogicalClient
> org.apache.derby.imp.drda.net.LogicalServer
> See the added paper for more thorough documentation on design choices and implementation along with some benchmarking results
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2852) Derby: Replication and Availability
Posted by "Daniel John Debrunner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12506901 ]
Daniel John Debrunner commented on DERBY-2852:
----------------------------------------------
Very interesting, I assume the diffs are against 10.2.2.0. Have you tried applying it to the trunk, that's where it will need to go if it gets committed.
Before the Derby project can accept this you need to file an ICLA with the ASF
These pages on the Derby wiki has information for getting involved in the Derby project.
http://wiki.apache.org/db-derby/DerbyDev
http://wiki.apache.org/db-derby/DerbyContributorChecklist
> Derby: Replication and Availability
> -----------------------------------
>
> Key: DERBY-2852
> URL: https://issues.apache.org/jira/browse/DERBY-2852
> Project: Derby
> Issue Type: New Feature
> Components: Network Server, Services, SQL, Store
> Affects Versions: 10.2.2.0
> Reporter: Egil Sørensen
> Attachments: derby-rep.zip, soereeg07, svn.diff
>
>
> This paper and attached source code describes the work done to add hot standby replication functionality to the Apache Derby Database Management System.
> By implementing a hot standby scheme in Apache Derby several features are added. The contents of the database is replicated at run time to another site providing online runtime backup. As the hot standby takes over on faults availability is added in that a client can connect to the hot standby after a crash. Thus the crash is masked from the clients. In addition to this, online upgrades of software and hardware can be done by taking down one database at the time. Then when
> the upgrade is completed the upgraded server is synchronized and back online with no downtime.
> A fully functional prototype of the Apache Derby hot standby scheme has been created in this project using logical logs, fail-fast takeovers and logical catchups after an internal up-to-crash recovery and reconnection. This project builds on the ideas that are presented in my earlier project: Derby: Write to Neighbor Mode.
> To enable the features described above some changes have been made to the Derby base system. The following classes have been changed or added:
> The Logical Log System
> org.apache.derby.iapi.store.logical.log.LogicalImage
> org.apache.derby.iapi.store.logical.log.LogicalLoggable
> org.apache.derby.impl.store.logical.log.LogicalLog
> org.apache.derby.impl.store.logical.log.LogicalLogger
> org.apache.derby.impl.store.logical.log.LogicalLogRecord
> org.apache.derby.impl.store.logical.opearations.LogicalAbortOperation
> org.apache.derby.impl.store.logical.opearations.LogicalCommitOperation
> org.apache.derby.impl.store.logical.opearations.LogicalDeleteOperation
> org.apache.derby.impl.store.logical.opearations.LogicalImage
> org.apache.derby.impl.store.logical.opearations.LogicalInsertOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperationFactory
> org.apache.derby.impl.store.logical.opearations.LogicalUpdateOperation
> The SQL Layer
> org.apache.impl.sql.execute.RowChangerImpl
> org.apache.impl.sql.conn.GenericLanguageConnectionContext
> The Communications Service
> org.apache.derby.impl.services.net.LogicalServer
> org.apache.derby.impl.services.net.LogicalClient
> org.apache.derby.impl.services.net.NetworkPayload
> org.apache.derby.impl.services.net.receiver.LogReceiver
> org.apache.derby.impl.services.net.shipper.LogShipper
> org.apache.derby.impl.services.net.shipper.LogicalCatchUp
> The Hot Standby Service
> org.apache.derby.impl.services.hotstandby.HotStandbyController
> org.apache.derby.impl.services.hotstandby.Database
> The Network Server
> org.apache.derby.impl.drda.NetworkServerControlImpl
> org.apache.derby.impl.drda.HotStandbyPoller
> org.apache.derby.impl.drda.net.LogicalClient
> org.apache.derby.imp.drda.net.LogicalServer
> See the added paper for more thorough documentation on design choices and implementation along with some benchmarking results
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2852) Derby: Replication and Availability
Posted by "Egil Sørensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Egil Sørensen updated DERBY-2852:
---------------------------------
Attachment: svn.diff
The svn diff from current release (10.2)
> Derby: Replication and Availability
> -----------------------------------
>
> Key: DERBY-2852
> URL: https://issues.apache.org/jira/browse/DERBY-2852
> Project: Derby
> Issue Type: New Feature
> Components: Network Server, Services, SQL, Store
> Affects Versions: 10.2.2.0
> Reporter: Egil Sørensen
> Attachments: derby-rep.zip, soereeg07, svn.diff
>
>
> This paper and attached source code describes the work done to add hot standby replication functionality to the Apache Derby Database Management System.
> By implementing a hot standby scheme in Apache Derby several features are added. The contents of the database is replicated at run time to another site providing online runtime backup. As the hot standby takes over on faults availability is added in that a client can connect to the hot standby after a crash. Thus the crash is masked from the clients. In addition to this, online upgrades of software and hardware can be done by taking down one database at the time. Then when
> the upgrade is completed the upgraded server is synchronized and back online with no downtime.
> A fully functional prototype of the Apache Derby hot standby scheme has been created in this project using logical logs, fail-fast takeovers and logical catchups after an internal up-to-crash recovery and reconnection. This project builds on the ideas that are presented in my earlier project: Derby: Write to Neighbor Mode.
> To enable the features described above some changes have been made to the Derby base system. The following classes have been changed or added:
> The Logical Log System
> org.apache.derby.iapi.store.logical.log.LogicalImage
> org.apache.derby.iapi.store.logical.log.LogicalLoggable
> org.apache.derby.impl.store.logical.log.LogicalLog
> org.apache.derby.impl.store.logical.log.LogicalLogger
> org.apache.derby.impl.store.logical.log.LogicalLogRecord
> org.apache.derby.impl.store.logical.opearations.LogicalAbortOperation
> org.apache.derby.impl.store.logical.opearations.LogicalCommitOperation
> org.apache.derby.impl.store.logical.opearations.LogicalDeleteOperation
> org.apache.derby.impl.store.logical.opearations.LogicalImage
> org.apache.derby.impl.store.logical.opearations.LogicalInsertOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperationFactory
> org.apache.derby.impl.store.logical.opearations.LogicalUpdateOperation
> The SQL Layer
> org.apache.impl.sql.execute.RowChangerImpl
> org.apache.impl.sql.conn.GenericLanguageConnectionContext
> The Communications Service
> org.apache.derby.impl.services.net.LogicalServer
> org.apache.derby.impl.services.net.LogicalClient
> org.apache.derby.impl.services.net.NetworkPayload
> org.apache.derby.impl.services.net.receiver.LogReceiver
> org.apache.derby.impl.services.net.shipper.LogShipper
> org.apache.derby.impl.services.net.shipper.LogicalCatchUp
> The Hot Standby Service
> org.apache.derby.impl.services.hotstandby.HotStandbyController
> org.apache.derby.impl.services.hotstandby.Database
> The Network Server
> org.apache.derby.impl.drda.NetworkServerControlImpl
> org.apache.derby.impl.drda.HotStandbyPoller
> org.apache.derby.impl.drda.net.LogicalClient
> org.apache.derby.imp.drda.net.LogicalServer
> See the added paper for more thorough documentation on design choices and implementation along with some benchmarking results
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2852) Derby: Replication and Availability
Posted by "Egil Sørensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Egil Sørensen updated DERBY-2852:
---------------------------------
Attachment: derby-rep.zip
Added and changed source code
> Derby: Replication and Availability
> -----------------------------------
>
> Key: DERBY-2852
> URL: https://issues.apache.org/jira/browse/DERBY-2852
> Project: Derby
> Issue Type: New Feature
> Components: Network Server, Services, SQL, Store
> Affects Versions: 10.2.2.0
> Reporter: Egil Sørensen
> Attachments: derby-rep.zip, soereeg07
>
>
> This paper and attached source code describes the work done to add hot standby replication functionality to the Apache Derby Database Management System.
> By implementing a hot standby scheme in Apache Derby several features are added. The contents of the database is replicated at run time to another site providing online runtime backup. As the hot standby takes over on faults availability is added in that a client can connect to the hot standby after a crash. Thus the crash is masked from the clients. In addition to this, online upgrades of software and hardware can be done by taking down one database at the time. Then when
> the upgrade is completed the upgraded server is synchronized and back online with no downtime.
> A fully functional prototype of the Apache Derby hot standby scheme has been created in this project using logical logs, fail-fast takeovers and logical catchups after an internal up-to-crash recovery and reconnection. This project builds on the ideas that are presented in my earlier project: Derby: Write to Neighbor Mode.
> To enable the features described above some changes have been made to the Derby base system. The following classes have been changed or added:
> The Logical Log System
> org.apache.derby.iapi.store.logical.log.LogicalImage
> org.apache.derby.iapi.store.logical.log.LogicalLoggable
> org.apache.derby.impl.store.logical.log.LogicalLog
> org.apache.derby.impl.store.logical.log.LogicalLogger
> org.apache.derby.impl.store.logical.log.LogicalLogRecord
> org.apache.derby.impl.store.logical.opearations.LogicalAbortOperation
> org.apache.derby.impl.store.logical.opearations.LogicalCommitOperation
> org.apache.derby.impl.store.logical.opearations.LogicalDeleteOperation
> org.apache.derby.impl.store.logical.opearations.LogicalImage
> org.apache.derby.impl.store.logical.opearations.LogicalInsertOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperation
> org.apache.derby.impl.store.logical.opearations.LogicalOperationFactory
> org.apache.derby.impl.store.logical.opearations.LogicalUpdateOperation
> The SQL Layer
> org.apache.impl.sql.execute.RowChangerImpl
> org.apache.impl.sql.conn.GenericLanguageConnectionContext
> The Communications Service
> org.apache.derby.impl.services.net.LogicalServer
> org.apache.derby.impl.services.net.LogicalClient
> org.apache.derby.impl.services.net.NetworkPayload
> org.apache.derby.impl.services.net.receiver.LogReceiver
> org.apache.derby.impl.services.net.shipper.LogShipper
> org.apache.derby.impl.services.net.shipper.LogicalCatchUp
> The Hot Standby Service
> org.apache.derby.impl.services.hotstandby.HotStandbyController
> org.apache.derby.impl.services.hotstandby.Database
> The Network Server
> org.apache.derby.impl.drda.NetworkServerControlImpl
> org.apache.derby.impl.drda.HotStandbyPoller
> org.apache.derby.impl.drda.net.LogicalClient
> org.apache.derby.imp.drda.net.LogicalServer
> See the added paper for more thorough documentation on design choices and implementation along with some benchmarking results
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.