Change in asterixdb[master]: Introducing Data Replication To AsterixDB

["Murtadha Hubail (Code Review)" <do...@asterixdb.incubator.apache.org>]

Hello abdullah alamoudi, Yingyi Bu, Jenkins,

I'd like you to reexamine a change.  Please visit

    https://asterix-gerrit.ics.uci.edu/338

to look at the new patch set (#5).

Change subject: Introducing Data Replication To AsterixDB
......................................................................

Introducing Data Replication To AsterixDB

This change includes the following:
- Add data replication properties to cluster properties and Managix validate command.
- Introduce Data Replication components.
- Add data replication required fields to LogRecord.
- Specialized LogManager for data replication.
- Fix for invalid cluster state on nodes failure.
- Fix for cleaning workspace files on startup/shutdown.
- Fix for temp datasets storage reclamation.
- Allow MetadataNode rebinding with CC.
- Add flag to checkpoint to identify sharp checkpoints.

Change-Id: I729fdd1144dbc9ff039b4bc414494860d7553810
---
M asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitRuntime.java
M asterix-app/pom.xml
M asterix-app/src/main/java/org/apache/asterix/api/common/AsterixAppRuntimeContext.java
M asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplicationEntryPoint.java
M asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/ClusterLifecycleListener.java
M asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
M asterix-common/pom.xml
M asterix-common/src/main/java/org/apache/asterix/common/api/IAsterixAppRuntimeContext.java
A asterix-common/src/main/java/org/apache/asterix/common/config/AsterixReplicationProperties.java
M asterix-common/src/main/java/org/apache/asterix/common/config/IAsterixPropertiesProvider.java
M asterix-common/src/main/java/org/apache/asterix/common/context/BaseOperationTracker.java
M asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
M asterix-common/src/main/java/org/apache/asterix/common/context/PrimaryIndexOperationTracker.java
M asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMIndexUtil.java
M asterix-common/src/main/java/org/apache/asterix/common/ioopcallbacks/AbstractLSMIOOperationCallback.java
M asterix-common/src/main/java/org/apache/asterix/common/ioopcallbacks/LSMBTreeIOOperationCallback.java
M asterix-common/src/main/java/org/apache/asterix/common/ioopcallbacks/LSMBTreeWithBuddyIOOperationCallback.java
M asterix-common/src/main/java/org/apache/asterix/common/ioopcallbacks/LSMInvertedIndexIOOperationCallback.java
M asterix-common/src/main/java/org/apache/asterix/common/ioopcallbacks/LSMRTreeIOOperationCallback.java
A asterix-common/src/main/java/org/apache/asterix/common/replication/AsterixReplicationJob.java
A asterix-common/src/main/java/org/apache/asterix/common/replication/IRemoteRecoveryManager.java
A asterix-common/src/main/java/org/apache/asterix/common/replication/IReplicaResourcesManager.java
A asterix-common/src/main/java/org/apache/asterix/common/replication/IReplicationChannel.java
A asterix-common/src/main/java/org/apache/asterix/common/replication/IReplicationLifecycleListener.java
A asterix-common/src/main/java/org/apache/asterix/common/replication/IReplicationManager.java
A asterix-common/src/main/java/org/apache/asterix/common/replication/IReplicationThread.java
A asterix-common/src/main/java/org/apache/asterix/common/replication/Replica.java
A asterix-common/src/main/java/org/apache/asterix/common/replication/ReplicaEvent.java
M asterix-common/src/main/java/org/apache/asterix/common/transactions/ILogManager.java
M asterix-common/src/main/java/org/apache/asterix/common/transactions/ILogPage.java
M asterix-common/src/main/java/org/apache/asterix/common/transactions/ILogRecord.java
M asterix-common/src/main/java/org/apache/asterix/common/transactions/IRecoveryManager.java
M asterix-common/src/main/java/org/apache/asterix/common/transactions/LogRecord.java
A asterix-common/src/main/java/org/apache/asterix/common/transactions/LogSource.java
M asterix-common/src/main/resources/schema/cluster.xsd
M asterix-events/src/main/java/org/apache/asterix/event/util/PatternCreator.java
M asterix-installer/src/main/java/org/apache/asterix/installer/command/ValidateCommand.java
M asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataManager.java
M asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlMetadataProvider.java
M asterix-om/src/main/java/org/apache/asterix/om/util/AsterixAppContextInfo.java
M asterix-om/src/main/java/org/apache/asterix/om/util/AsterixClusterProperties.java
A asterix-replication/pom.xml
A asterix-replication/src/main/java/org/apache/asterix/replication/functions/AsterixReplicationProtocol.java
A asterix-replication/src/main/java/org/apache/asterix/replication/functions/ReplicaFilesRequest.java
A asterix-replication/src/main/java/org/apache/asterix/replication/functions/ReplicaIndexFlushRequest.java
A asterix-replication/src/main/java/org/apache/asterix/replication/functions/ReplicaLogsRequest.java
A asterix-replication/src/main/java/org/apache/asterix/replication/logging/RemoteLogMapping.java
A asterix-replication/src/main/java/org/apache/asterix/replication/management/NetworkingUtil.java
A asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicaEventNotifier.java
A asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicaStateChecker.java
A asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicationChannel.java
A asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicationLifecycleListener.java
A asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicationManager.java
A asterix-replication/src/main/java/org/apache/asterix/replication/recovery/RemoteRecoveryManager.java
A asterix-replication/src/main/java/org/apache/asterix/replication/storage/AsterixFilesUtil.java
A asterix-replication/src/main/java/org/apache/asterix/replication/storage/AsterixLSMIndexFileProperties.java
A asterix-replication/src/main/java/org/apache/asterix/replication/storage/LSMComponentLSNSyncTask.java
A asterix-replication/src/main/java/org/apache/asterix/replication/storage/LSMComponentProperties.java
A asterix-replication/src/main/java/org/apache/asterix/replication/storage/ReplicaResourcesManager.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/opcallbacks/AbstractIndexModificationOperationCallback.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/locking/ConcurrentLockManager.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/logging/LogManager.java
A asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/logging/LogManagerWithReplication.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/logging/LogPage.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/recovery/CheckpointObject.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/recovery/RecoveryManager.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/transaction/TransactionContext.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/transaction/TransactionSubsystem.java
M pom.xml
70 files changed, 6,512 insertions(+), 540 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/38/338/5
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/338
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I729fdd1144dbc9ff039b4bc414494860d7553810
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <hu...@gmail.com>
Gerrit-Reviewer: Ian Maxon <im...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Yingyi Bu <bu...@gmail.com>
Gerrit-Reviewer: Young-Seok Kim <ki...@gmail.com>
Gerrit-Reviewer: abdullah alamoudi <ba...@gmail.com>