You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by du...@apache.org on 2022/07/21 01:23:52 UTC
[rocketmq] branch develop updated (1905756db -> efbc4a1fc)
This is an automated email from the ASF dual-hosted git repository.
duhengforever pushed a change to branch develop
in repository https://gitbox.apache.org/repos/asf/rocketmq.git
from 1905756db [ISSUE #4062] pop consumer does not require static topic
add bcce3d32c Fix check style to paas CI
add d26773a9a [Summer of Code] Dledger controller implementation (#4195)
add 911ee340c [Summer of Code] Support switch role for ha service (#4236)
add 541a6d012 Merge remote-tracking branch 'apache/5.0.0-beta' into 5.0.0-beta-dledger-controller
add ce534eea1 Update some service name in ha service
add ad8851bef [Summer of Code] Support switch role for broker (#4272)
add 2f6da9e77 Make thread pool in ReplicasManager can distinguish different brokers
add 4e9e4e3ef [Summer of code] Stand alone a new controller module (#4333)
add f905da411 [Summer of code] Let broker send heartbeat to controller (#4341)
add f5c5f5c4f [Summer of code] Shrink and expand InSyncStateSet (#4355)
add 25ee6eb32 [Summer of code] Support async learner in controller mode (#4367)
add 97e6cecbf [Summer of code] Fix some bugs in controller mode.
add e0121ca1d [Summer of code] Add admin tools for controller mode (#4388)
add 0efbb3592 Polish the admin command naming convention
add 6f3b372e8 Polish naming convention of controller
add 99d77df54 [Summer of code] Fix the isuue of frequent disconnection of ha connection (#4399)
add c4a7acdc0 [Summer of code] Fix allAckSyncStateSet not working
add addd307e0 Remove useless remotingClient in ControllerManager
add 12d9207d4 [Summer of code] Add sh conf for controller
add 05841188e [Summer of code] Add admin tool getControllerMetadata (#4405)
add f27800a13 Polish switching logic and auto switch ha code (#4406)
add 8515a8d09 [Summer of code] Reuse dledger remotingServer in controller mode. (#4409)
add 57be558d7 [Summer of code] Record haconnection's lastCaughtUpTimeMs in haService's map (#4414)
add 55a0e7b88 [Summer of code] Add design and quick_start document for controller mode (#4413)
add 3e0a41d4c Polish the mq controller document
add e7bf51f4c Fix ha controller bugs (#4422)
add fa8ce658c Ensure that new messages cannot be written during failover (#4423)
add 98a283f5a [Summer of code] Let controller inform broker that role changed. (#4424)
add ebce38705 Add notifyBrokerRoleChanged configuration
add 9094de9a6 Polish the GetSyncStateSetSubCommand
add 81dc9386a Make broker do not online before get brokerId and master-slave relationship (#4433)
add 6645d0a42 [Summer of code] Let controller become role state after append initial logs (#4442)
add 5eaeeabf5 [Summer of code] Let broker register to controller again when master not existed. (#4450)
add f038cf95d [Summer of code] Use confirm offset in ReputMessageService (#4449)
add d9273e70f Update the logic of confirmOffset and recover topicQueueTable to prevent message loss when broker role change (#4460)
add fd804d9a9 Polish the logic of change to master (#4464)
add 3fbcad5f9 Remove allAckInSyncStateSet in GroupCommitRequest and use ackNums = -1 instead (#4467)
add c2b21e4e2 Some minor document fixes
add d6e922290 Update hacontroller directory name to controller
add fd8dafd5f ReplicasInfoManager use controller config instead of enableElectUncleanMaster so that it can be modified at run time
add 9c842e92a review (#4475)
add a8549e24a Add deploy and upgrade guide document for controller mode (#4476)
add c58a022bc Remove debug log output
add 555026676 1.Rename BrokerRegisterRequestHeader to RegisterBrokerToControllerRequestHeader 2.Make ChangeToMaster do all things even if it is already master
add 1b77be4d2 Add back haConnectionStateNotificationService to HAService
add 505cb2af9 Merge branch '5.0.0-beta' into 5.0.0-beta-dledger-controller
add e5c538a25 When the size of syncStateSet is less than minInSyncReplicas, put message returns PutMessageStatus.IN_SYNC_REPLICAS_NOT_ENOUGH
add c3a3334e8 When enableSlaveActingMaster=false, automatic degradation is not performed
add dde98ac2c Merge remote-tracking branch 'apache/5.0.0-beta' into 5.0.0-beta-dledger-controller
add 3d8296e3b When the controller is embedded in namesrv, the heartbeat of namesrv is no longer reused
add ebafd7de7 Fix bug that CI can not pass if function interface changed
add a55a62159 Merge branch 'develop-merge-5.0.0' into 5.0.0-beta-dledger-controller
add ab4f7ddbc Pass the CI after merge develop
add 4be6e2c1e Upgrade dledger version to 0.2.7
add d47f7024f Polish some pom.xml file and remove useless dependency
add 9dd34df41 Merge branch '5.0.0-beta-dledger-controller' of github.com:apache/rocketmq into 5.0.0-beta-dledger-controller
add b6fac7444 Merge branch 'develop-merge-5.0.0' into 5.0.0-beta-dledger-controller
add e09591486 Change rocketmq version to 5.0.0-SNAPSHOT in controller pom.xml
add 8dfa8813c Merge remote-tracking branch 'apache/develop' into 5.0.0-beta-dledger-controller
add ed8922cdd 1. Add back main0 function in NamesrvStartup for compatibility 2. Make changeToMaster and changeToSlave have default implementation
add 67a526211 Use ${dledger.version} instead of fixed version in pom file
add cc4c19642 Change the loading order of HAService
add eb4e68464 [ISSUE #4626] Add CRC32 verification when saving checkpoint file (#4627)
add dd09f2b78 Merge remote-tracking branch 'apache/develop' into 5.0.0-beta-dledger-controller
add 9711f3b11 Make replicasManager shutdown when brokerController shutdown
add 86568e2d6 Remove test scope to pass CI
new efbc4a1fc Merge pull request #4484 from apache/5.0.0-beta-dledger-controller
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../apache/rocketmq/broker/BrokerController.java | 126 ++--
.../org/apache/rocketmq/broker/BrokerStartup.java | 28 +-
.../broker/controller/ReplicasManager.java | 447 +++++++++++++
.../apache/rocketmq/broker/out/BrokerOuterAPI.java | 145 ++++-
.../broker/processor/AdminBrokerProcessor.java | 68 +-
.../rocketmq/client/impl/MQClientAPIImpl.java | 52 ++
.../org/apache/rocketmq/common/BrokerAddrInfo.java | 78 +++
.../org/apache/rocketmq/common/BrokerConfig.java | 63 +-
.../apache/rocketmq/common/ControllerConfig.java | 178 ++++++
.../org/apache/rocketmq/common/EpochEntry.java | 91 +++
.../java/org/apache/rocketmq/common/MixAll.java | 1 +
.../rocketmq/common/constant/LoggerName.java | 1 +
.../rocketmq/common/namesrv/NamesrvConfig.java | 2 +-
.../rocketmq/common/protocol/RequestCode.java | 19 +
.../rocketmq/common/protocol/ResponseCode.java | 11 +
...terAclVersionInfo.java => EpochEntryCache.java} | 70 +-
.../common/protocol/body/InSyncStateData.java | 124 ++++
.../body/{TopicList.java => SyncStateSet.java} | 35 +-
.../NotifyBrokerRoleChangedRequestHeader.java | 83 +++
.../AlterSyncStateSetRequestHeader.java} | 55 +-
.../AlterSyncStateSetResponseHeader.java} | 31 +-
.../controller/ElectMasterRequestHeader.java} | 31 +-
.../controller/ElectMasterResponseHeader.java | 76 +++
.../controller/GetMetaDataResponseHeader.java | 93 +++
.../GetReplicaInfoRequestHeader.java} | 52 +-
.../controller/GetReplicaInfoResponseHeader.java} | 54 +-
.../RegisterBrokerToControllerRequestHeader.java} | 63 +-
.../RegisterBrokerToControllerResponseHeader.java} | 62 +-
.../rocketmq/common/utils/CheckpointFile.java | 155 +++++
.../rocketmq/common/utils/FastJsonSerializer.java | 62 ++
.../{CorrelationIdUtil.java => Serializer.java} | 21 +-
.../rocketmq/common/utils/CheckpointFileTest.java | 97 +++
.../container/BrokerContainerProcessor.java | 30 +-
.../rocketmq/container/BrokerContainerStartup.java | 28 +-
.../rocketmq/container/InnerBrokerController.java | 19 +-
{store => controller}/pom.xml | 21 +-
.../controller/BrokerHeartbeatManager.java | 70 ++
.../controller}/BrokerHousekeepingService.java | 15 +-
.../org/apache/rocketmq/controller/Controller.java | 109 ++++
.../rocketmq/controller/ControllerManager.java | 220 +++++++
.../rocketmq/controller/ControllerStartup.java | 60 +-
.../controller/impl/DLedgerController.java | 455 +++++++++++++
.../impl/DLedgerControllerStateMachine.java | 75 +++
.../impl/DefaultBrokerHeartbeatManager.java | 188 ++++++
.../impl/event/AlterSyncStateSetEvent.java | 42 +-
.../controller/impl/event/ApplyBrokerIdEvent.java | 58 ++
.../controller/impl/event/ControllerResult.java | 87 +++
.../controller/impl/event/ElectMasterEvent.java | 74 +++
.../controller/impl/event/EventMessage.java | 15 +-
.../controller/impl/event/EventSerializer.java | 77 +++
.../rocketmq/controller/impl/event/EventType.java | 49 +-
.../controller/impl/manager/BrokerInfo.java | 75 +++
.../impl/manager/ReplicasInfoManager.java | 421 ++++++++++++
.../controller/impl/manager/SyncStateInfo.java | 82 +++
.../processor/ControllerRequestProcessor.java | 143 +++++
.../impl/controller/ControllerManagerTest.java | 196 ++++++
.../controller/impl/DLedgerControllerTest.java | 280 ++++++++
.../impl/DefaultBrokerHeartbeatManagerTest.java | 52 ++
.../impl/manager/ReplicasInfoManagerTest.java | 141 ++++
.../src/test/resources/logback-test.xml | 0
.../bin/{dledger => controller}/fast-try.sh | 19 +-
.../bin/{mqbroker.numanode0 => mqcontroller} | 4 +-
distribution/bin/{mqproxy.cmd => mqcontroller.cmd} | 4 +-
distribution/bin/mqshutdown | 16 +-
distribution/bin/mqshutdown.cmd | 8 +-
.../conf/controller/quick-start/broker-n0.conf | 28 +-
.../conf/controller/quick-start/broker-n1.conf | 28 +-
.../conf/controller/quick-start/namesrv.conf | 19 +-
...{logback_namesrv.xml => logback_controller.xml} | 26 +-
docs/cn/controller/deploy.md | 93 +++
docs/cn/controller/design.md | 201 ++++++
docs/cn/controller/quick_start.md | 80 +++
docs/cn/image/controller/controller_design_1.png | Bin 0 -> 427708 bytes
.../image/controller/quick-start/changemaster.png | Bin 0 -> 13598 bytes
.../cn/image/controller/quick-start/controller.png | Bin 0 -> 315128 bytes
docs/cn/image/controller/quick-start/epoch.png | Bin 0 -> 14409 bytes
.../image/controller/quick-start/syncstateset.png | Bin 0 -> 13030 bytes
namesrv/pom.xml | 4 +
.../apache/rocketmq/namesrv/NamesrvStartup.java | 97 ++-
.../namesrv/routeinfo/RouteInfoManager.java | 65 +-
pom.xml | 11 +
.../remoting/netty/NettyRemotingClient.java | 2 +-
.../java/org/apache/rocketmq/store/CommitLog.java | 45 +-
.../apache/rocketmq/store/DefaultMessageStore.java | 49 +-
.../org/apache/rocketmq/store/MessageStore.java | 41 +-
.../rocketmq/store/config/MessageStoreConfig.java | 65 +-
.../apache/rocketmq/store/ha/DefaultHAService.java | 62 +-
.../rocketmq/store/ha/GroupTransferService.java | 48 +-
.../org/apache/rocketmq/store/ha/HAService.java | 28 +-
.../store/ha/autoswitch/AutoSwitchHAClient.java | 526 +++++++++++++++
.../ha/autoswitch/AutoSwitchHAConnection.java | 708 +++++++++++++++++++++
.../store/ha/autoswitch/AutoSwitchHAService.java | 407 ++++++++++++
.../store/ha/autoswitch/EpochFileCache.java | 327 ++++++++++
.../rocketmq/store/ha/io/AbstractHAReader.java | 80 +++
.../apache/rocketmq/store/ha/io/HAReadHook.java | 5 +-
.../apache/rocketmq/store/ha/io/HAWriteHook.java | 5 +-
.../org/apache/rocketmq/store/ha/io/HAWriter.java | 61 ++
.../org/apache/rocketmq/store/ha/HAServerTest.java | 21 +-
.../store/ha/autoswitch/AutoSwitchHATest.java | 505 +++++++++++++++
.../store/ha/autoswitch/EpochFileCacheTest.java | 149 +++++
.../test/autoswitchrole/AutoSwitchRoleBase.java | 178 ++++++
.../AutoSwitchRoleIntegrationTest.java | 239 +++++++
.../test/container/PullMultipleReplicasIT.java | 2 +-
.../rocketmq/test/container/SlaveBrokerIT.java | 2 +-
.../rocketmq/tools/admin/DefaultMQAdminExt.java | 30 +-
.../tools/admin/DefaultMQAdminExtImpl.java | 18 +
.../apache/rocketmq/tools/admin/MQAdminExt.java | 19 +-
.../apache/rocketmq/tools/command/CommandUtil.java | 9 +
.../rocketmq/tools/command/MQAdminStartup.java | 15 +-
.../command/broker/GetBrokerEpochCommand.java | 119 ++++
.../GetControllerMetaDataCommand.java} | 38 +-
.../command/ha/GetSyncStateSetSubCommand.java | 130 ++++
.../tools/command/ha/HAStatusSubCommand.java | 10 +-
113 files changed, 9466 insertions(+), 736 deletions(-)
create mode 100644 broker/src/main/java/org/apache/rocketmq/broker/controller/ReplicasManager.java
create mode 100644 common/src/main/java/org/apache/rocketmq/common/BrokerAddrInfo.java
create mode 100644 common/src/main/java/org/apache/rocketmq/common/ControllerConfig.java
create mode 100644 common/src/main/java/org/apache/rocketmq/common/EpochEntry.java
copy common/src/main/java/org/apache/rocketmq/common/protocol/body/{ClusterAclVersionInfo.java => EpochEntryCache.java} (51%)
create mode 100644 common/src/main/java/org/apache/rocketmq/common/protocol/body/InSyncStateData.java
copy common/src/main/java/org/apache/rocketmq/common/protocol/body/{TopicList.java => SyncStateSet.java} (52%)
create mode 100644 common/src/main/java/org/apache/rocketmq/common/protocol/header/NotifyBrokerRoleChangedRequestHeader.java
copy common/src/main/java/org/apache/rocketmq/common/protocol/header/namesrv/{UnRegisterBrokerRequestHeader.java => controller/AlterSyncStateSetRequestHeader.java} (54%)
copy common/src/main/java/org/apache/rocketmq/common/protocol/header/{InitConsumerOffsetRequestHeader.java => namesrv/controller/AlterSyncStateSetResponseHeader.java} (64%)
copy common/src/main/java/org/apache/rocketmq/common/protocol/header/{filtersrv/RegisterFilterServerResponseHeader.java => namesrv/controller/ElectMasterRequestHeader.java} (73%)
create mode 100644 common/src/main/java/org/apache/rocketmq/common/protocol/header/namesrv/controller/ElectMasterResponseHeader.java
create mode 100644 common/src/main/java/org/apache/rocketmq/common/protocol/header/namesrv/controller/GetMetaDataResponseHeader.java
copy common/src/main/java/org/apache/rocketmq/common/protocol/header/namesrv/{QueryDataVersionRequestHeader.java => controller/GetReplicaInfoRequestHeader.java} (59%)
copy common/src/main/java/org/apache/rocketmq/common/protocol/header/{ExchangeHAInfoResponseHeader.java => namesrv/controller/GetReplicaInfoResponseHeader.java} (59%)
copy common/src/main/java/org/apache/rocketmq/common/protocol/header/namesrv/{QueryDataVersionRequestHeader.java => controller/RegisterBrokerToControllerRequestHeader.java} (53%)
copy common/src/main/java/org/apache/rocketmq/common/protocol/header/{ExchangeHAInfoRequestHeader.java => namesrv/controller/RegisterBrokerToControllerResponseHeader.java} (51%)
create mode 100644 common/src/main/java/org/apache/rocketmq/common/utils/CheckpointFile.java
create mode 100644 common/src/main/java/org/apache/rocketmq/common/utils/FastJsonSerializer.java
copy common/src/main/java/org/apache/rocketmq/common/utils/{CorrelationIdUtil.java => Serializer.java} (70%)
create mode 100644 common/src/test/java/org/apache/rocketmq/common/utils/CheckpointFileTest.java
copy {store => controller}/pom.xml (86%)
create mode 100644 controller/src/main/java/org/apache/rocketmq/controller/BrokerHeartbeatManager.java
copy {namesrv/src/main/java/org/apache/rocketmq/namesrv/routeinfo => controller/src/main/java/org/apache/rocketmq/controller}/BrokerHousekeepingService.java (75%)
create mode 100644 controller/src/main/java/org/apache/rocketmq/controller/Controller.java
create mode 100644 controller/src/main/java/org/apache/rocketmq/controller/ControllerManager.java
copy namesrv/src/main/java/org/apache/rocketmq/namesrv/NamesrvStartup.java => controller/src/main/java/org/apache/rocketmq/controller/ControllerStartup.java (69%)
create mode 100644 controller/src/main/java/org/apache/rocketmq/controller/impl/DLedgerController.java
create mode 100644 controller/src/main/java/org/apache/rocketmq/controller/impl/DLedgerControllerStateMachine.java
create mode 100644 controller/src/main/java/org/apache/rocketmq/controller/impl/DefaultBrokerHeartbeatManager.java
copy common/src/main/java/org/apache/rocketmq/common/protocol/header/GetBrokerMemberGroupRequestHeader.java => controller/src/main/java/org/apache/rocketmq/controller/impl/event/AlterSyncStateSetEvent.java (51%)
create mode 100644 controller/src/main/java/org/apache/rocketmq/controller/impl/event/ApplyBrokerIdEvent.java
create mode 100644 controller/src/main/java/org/apache/rocketmq/controller/impl/event/ControllerResult.java
create mode 100644 controller/src/main/java/org/apache/rocketmq/controller/impl/event/ElectMasterEvent.java
copy common/src/main/java/org/apache/rocketmq/common/statistics/Interceptor.java => controller/src/main/java/org/apache/rocketmq/controller/impl/event/EventMessage.java (78%)
create mode 100644 controller/src/main/java/org/apache/rocketmq/controller/impl/event/EventSerializer.java
copy common/src/main/java/org/apache/rocketmq/common/statistics/StatisticsKindMeta.java => controller/src/main/java/org/apache/rocketmq/controller/impl/event/EventType.java (52%)
create mode 100644 controller/src/main/java/org/apache/rocketmq/controller/impl/manager/BrokerInfo.java
create mode 100644 controller/src/main/java/org/apache/rocketmq/controller/impl/manager/ReplicasInfoManager.java
create mode 100644 controller/src/main/java/org/apache/rocketmq/controller/impl/manager/SyncStateInfo.java
create mode 100644 controller/src/main/java/org/apache/rocketmq/controller/processor/ControllerRequestProcessor.java
create mode 100644 controller/src/test/java/org/apache/rocketmq/controller/impl/controller/ControllerManagerTest.java
create mode 100644 controller/src/test/java/org/apache/rocketmq/controller/impl/controller/impl/DLedgerControllerTest.java
create mode 100644 controller/src/test/java/org/apache/rocketmq/controller/impl/controller/impl/DefaultBrokerHeartbeatManagerTest.java
create mode 100644 controller/src/test/java/org/apache/rocketmq/controller/impl/controller/impl/manager/ReplicasInfoManagerTest.java
copy {test => controller}/src/test/resources/logback-test.xml (100%)
copy distribution/bin/{dledger => controller}/fast-try.sh (76%)
copy distribution/bin/{mqbroker.numanode0 => mqcontroller} (93%)
copy distribution/bin/{mqproxy.cmd => mqcontroller.cmd} (88%)
copy broker/src/test/resources/logback-test.xml => distribution/conf/controller/quick-start/broker-n0.conf (63%)
copy broker/src/test/resources/logback-test.xml => distribution/conf/controller/quick-start/broker-n1.conf (63%)
copy broker/src/test/resources/logback-test.xml => distribution/conf/controller/quick-start/namesrv.conf (63%)
copy distribution/conf/{logback_namesrv.xml => logback_controller.xml} (80%)
create mode 100644 docs/cn/controller/deploy.md
create mode 100644 docs/cn/controller/design.md
create mode 100644 docs/cn/controller/quick_start.md
create mode 100644 docs/cn/image/controller/controller_design_1.png
create mode 100644 docs/cn/image/controller/quick-start/changemaster.png
create mode 100644 docs/cn/image/controller/quick-start/controller.png
create mode 100644 docs/cn/image/controller/quick-start/epoch.png
create mode 100644 docs/cn/image/controller/quick-start/syncstateset.png
create mode 100644 store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/AutoSwitchHAClient.java
create mode 100644 store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/AutoSwitchHAConnection.java
create mode 100644 store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/AutoSwitchHAService.java
create mode 100644 store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/EpochFileCache.java
create mode 100644 store/src/main/java/org/apache/rocketmq/store/ha/io/AbstractHAReader.java
copy acl/src/main/java/org/apache/rocketmq/acl/AccessResource.java => store/src/main/java/org/apache/rocketmq/store/ha/io/HAReadHook.java (88%)
copy acl/src/main/java/org/apache/rocketmq/acl/AccessResource.java => store/src/main/java/org/apache/rocketmq/store/ha/io/HAWriteHook.java (88%)
create mode 100644 store/src/main/java/org/apache/rocketmq/store/ha/io/HAWriter.java
create mode 100644 store/src/test/java/org/apache/rocketmq/store/ha/autoswitch/AutoSwitchHATest.java
create mode 100644 store/src/test/java/org/apache/rocketmq/store/ha/autoswitch/EpochFileCacheTest.java
create mode 100644 test/src/test/java/org/apache/rocketmq/test/autoswitchrole/AutoSwitchRoleBase.java
create mode 100644 test/src/test/java/org/apache/rocketmq/test/autoswitchrole/AutoSwitchRoleIntegrationTest.java
create mode 100644 tools/src/main/java/org/apache/rocketmq/tools/command/broker/GetBrokerEpochCommand.java
copy tools/src/main/java/org/apache/rocketmq/tools/command/{container/AddBrokerSubCommand.java => controller/GetControllerMetaDataCommand.java} (59%)
create mode 100644 tools/src/main/java/org/apache/rocketmq/tools/command/ha/GetSyncStateSetSubCommand.java
[rocketmq] 01/01: Merge pull request #4484 from apache/5.0.0-beta-dledger-controller
Posted by du...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
duhengforever pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/rocketmq.git
commit efbc4a1fccd2e79b339ee2e6da7695a03eb31139
Merge: 1905756db 86568e2d6
Author: Heng Du <du...@apache.org>
AuthorDate: Thu Jul 21 09:23:27 2022 +0800
Merge pull request #4484 from apache/5.0.0-beta-dledger-controller
[RIP-44] Support DLedger Controller
.../apache/rocketmq/broker/BrokerController.java | 126 ++--
.../org/apache/rocketmq/broker/BrokerStartup.java | 28 +-
.../broker/controller/ReplicasManager.java | 447 +++++++++++++
.../apache/rocketmq/broker/out/BrokerOuterAPI.java | 145 ++++-
.../broker/processor/AdminBrokerProcessor.java | 68 +-
.../rocketmq/client/impl/MQClientAPIImpl.java | 52 ++
.../org/apache/rocketmq/common/BrokerAddrInfo.java | 78 +++
.../org/apache/rocketmq/common/BrokerConfig.java | 63 +-
.../apache/rocketmq/common/ControllerConfig.java | 178 ++++++
.../org/apache/rocketmq/common/EpochEntry.java | 91 +++
.../java/org/apache/rocketmq/common/MixAll.java | 1 +
.../rocketmq/common/constant/LoggerName.java | 1 +
.../rocketmq/common/namesrv/NamesrvConfig.java | 2 +-
.../rocketmq/common/protocol/RequestCode.java | 19 +
.../rocketmq/common/protocol/ResponseCode.java | 11 +
.../common/protocol/body/EpochEntryCache.java | 87 +++
.../common/protocol/body/InSyncStateData.java | 124 ++++
.../common/protocol/body/SyncStateSet.java | 55 ++
.../NotifyBrokerRoleChangedRequestHeader.java | 83 +++
.../controller/AlterSyncStateSetRequestHeader.java | 71 +++
.../AlterSyncStateSetResponseHeader.java | 45 ++
.../controller/ElectMasterRequestHeader.java | 50 ++
.../controller/ElectMasterResponseHeader.java | 76 +++
.../controller/GetMetaDataResponseHeader.java | 93 +++
.../controller/GetReplicaInfoRequestHeader.java | 64 ++
.../controller/GetReplicaInfoResponseHeader.java | 66 ++
.../RegisterBrokerToControllerRequestHeader.java | 83 +++
.../RegisterBrokerToControllerResponseHeader.java | 76 +++
.../rocketmq/common/utils/CheckpointFile.java | 155 +++++
.../rocketmq/common/utils/FastJsonSerializer.java | 62 ++
.../apache/rocketmq/common/utils/Serializer.java | 35 +
.../rocketmq/common/utils/CheckpointFileTest.java | 97 +++
.../container/BrokerContainerProcessor.java | 30 +-
.../rocketmq/container/BrokerContainerStartup.java | 28 +-
.../rocketmq/container/InnerBrokerController.java | 19 +-
{namesrv => controller}/pom.xml | 54 +-
.../controller/BrokerHeartbeatManager.java | 70 ++
.../controller/BrokerHousekeepingService.java | 51 ++
.../org/apache/rocketmq/controller/Controller.java | 109 ++++
.../rocketmq/controller/ControllerManager.java | 220 +++++++
.../rocketmq/controller/ControllerStartup.java | 60 +-
.../controller/impl/DLedgerController.java | 455 +++++++++++++
.../impl/DLedgerControllerStateMachine.java | 75 +++
.../impl/DefaultBrokerHeartbeatManager.java | 188 ++++++
.../impl/event/AlterSyncStateSetEvent.java | 55 ++
.../controller/impl/event/ApplyBrokerIdEvent.java | 58 ++
.../controller/impl/event/ControllerResult.java | 87 +++
.../controller/impl/event/ElectMasterEvent.java | 74 +++
.../controller/impl/event/EventMessage.java | 28 +
.../controller/impl/event/EventSerializer.java | 77 +++
.../rocketmq/controller/impl/event/EventType.java | 57 ++
.../controller/impl/manager/BrokerInfo.java | 75 +++
.../impl/manager/ReplicasInfoManager.java | 421 ++++++++++++
.../controller/impl/manager/SyncStateInfo.java | 82 +++
.../processor/ControllerRequestProcessor.java | 143 +++++
.../impl/controller/ControllerManagerTest.java | 196 ++++++
.../controller/impl/DLedgerControllerTest.java | 280 ++++++++
.../impl/DefaultBrokerHeartbeatManagerTest.java | 52 ++
.../impl/manager/ReplicasInfoManagerTest.java | 141 ++++
controller/src/test/resources/logback-test.xml | 33 +
distribution/bin/controller/fast-try.sh | 101 +++
distribution/bin/mqcontroller | 45 ++
.../bin/{mqshutdown.cmd => mqcontroller.cmd} | 20 +-
distribution/bin/mqshutdown | 16 +-
distribution/bin/mqshutdown.cmd | 8 +-
.../conf/controller/quick-start/broker-n0.conf | 30 +
.../conf/controller/quick-start/broker-n1.conf | 30 +
.../conf/controller/quick-start/namesrv.conf | 21 +
distribution/conf/logback_controller.xml | 94 +++
docs/cn/controller/deploy.md | 93 +++
docs/cn/controller/design.md | 201 ++++++
docs/cn/controller/quick_start.md | 80 +++
docs/cn/image/controller/controller_design_1.png | Bin 0 -> 427708 bytes
.../image/controller/quick-start/changemaster.png | Bin 0 -> 13598 bytes
.../cn/image/controller/quick-start/controller.png | Bin 0 -> 315128 bytes
docs/cn/image/controller/quick-start/epoch.png | Bin 0 -> 14409 bytes
.../image/controller/quick-start/syncstateset.png | Bin 0 -> 13030 bytes
namesrv/pom.xml | 4 +
.../apache/rocketmq/namesrv/NamesrvStartup.java | 97 ++-
.../namesrv/routeinfo/RouteInfoManager.java | 65 +-
pom.xml | 11 +
.../remoting/netty/NettyRemotingClient.java | 2 +-
.../java/org/apache/rocketmq/store/CommitLog.java | 45 +-
.../apache/rocketmq/store/DefaultMessageStore.java | 49 +-
.../org/apache/rocketmq/store/MessageStore.java | 41 +-
.../rocketmq/store/config/MessageStoreConfig.java | 65 +-
.../apache/rocketmq/store/ha/DefaultHAService.java | 62 +-
.../rocketmq/store/ha/GroupTransferService.java | 48 +-
.../org/apache/rocketmq/store/ha/HAService.java | 28 +-
.../store/ha/autoswitch/AutoSwitchHAClient.java | 526 +++++++++++++++
.../ha/autoswitch/AutoSwitchHAConnection.java | 708 +++++++++++++++++++++
.../store/ha/autoswitch/AutoSwitchHAService.java | 407 ++++++++++++
.../store/ha/autoswitch/EpochFileCache.java | 327 ++++++++++
.../rocketmq/store/ha/io/AbstractHAReader.java | 80 +++
.../apache/rocketmq/store/ha/io/HAReadHook.java | 22 +
.../apache/rocketmq/store/ha/io/HAWriteHook.java | 22 +
.../org/apache/rocketmq/store/ha/io/HAWriter.java | 61 ++
.../org/apache/rocketmq/store/ha/HAServerTest.java | 21 +-
.../store/ha/autoswitch/AutoSwitchHATest.java | 505 +++++++++++++++
.../store/ha/autoswitch/EpochFileCacheTest.java | 149 +++++
.../test/autoswitchrole/AutoSwitchRoleBase.java | 178 ++++++
.../AutoSwitchRoleIntegrationTest.java | 239 +++++++
.../test/container/PullMultipleReplicasIT.java | 2 +-
.../rocketmq/test/container/SlaveBrokerIT.java | 2 +-
.../rocketmq/tools/admin/DefaultMQAdminExt.java | 30 +-
.../tools/admin/DefaultMQAdminExtImpl.java | 18 +
.../apache/rocketmq/tools/admin/MQAdminExt.java | 19 +-
.../apache/rocketmq/tools/command/CommandUtil.java | 9 +
.../rocketmq/tools/command/MQAdminStartup.java | 15 +-
.../command/broker/GetBrokerEpochCommand.java | 119 ++++
.../controller/GetControllerMetaDataCommand.java | 68 ++
.../command/ha/GetSyncStateSetSubCommand.java | 130 ++++
.../tools/command/ha/HAStatusSubCommand.java | 10 +-
113 files changed, 10368 insertions(+), 405 deletions(-)