You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2023/03/15 09:19:29 UTC
[GitHub] [rocketmq] weihubeats created a discussion: 自动主从切换部署不能选出master
GitHub user weihubeats created a discussion: 自动主从切换部署不能选出master
nameserve: 2个节点
broker: 2个节点
错误log
2023-03-15 16:59:04 INFO ReplicasManager_ScheduledService_1 - Update controller leader address to x.x.x.1:9878
2023-03-15 16:59:06 WARN ReplicasManager_ExecutorService_1 - Failed to start replicasManager, retry times:35, current state:FIRST_TIME_SYNC_CONTROLLER_METADATA_DONE, try it again
2023-03-15 16:59:06 ERROR ReplicasManager_ExecutorService_1 - Failed to register broker to controller
org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2012 DESC: The broker has not master, and this new registered broker can't not be elected as master
For more information, please visit the url, https://rocketmq.apache.org/docs/bestPractice/06FAQ
at org.apache.rocketmq.broker.out.BrokerOuterAPI.registerBrokerToController(BrokerOuterAPI.java:1183)
at org.apache.rocketmq.broker.controller.ReplicasManager.registerBrokerToController(ReplicasManager.java:302)
at org.apache.rocketmq.broker.controller.ReplicasManager.startBasicService(ReplicasManager.java:151)
at org.apache.rocketmq.broker.controller.ReplicasManager.lambda$start$0(ReplicasManager.java:133)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2023-03-15 16:59:11 WARN ReplicasManager_ExecutorService_1 - Failed to start replicasManager, retry times:36, current state:FIRST_TIME_SYNC_CONTROLLER_METADATA_DONE, try it again
2023-03-15 16:59:11 ERROR ReplicasManager_ExecutorService_1 - Failed to register broker to controller
org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2012 DESC: The broker has not master, and this new registered broker can't not be elected as master
For more information, please visit the url, https://rocketmq.apache.org/docs/bestPractice/06FAQ
at org.apache.rocketmq.broker.out.BrokerOuterAPI.registerBrokerToController(BrokerOuterAPI.java:1183)
at org.apache.rocketmq.broker.controller.ReplicasManager.registerBrokerToController(ReplicasManager.java:302)
at org.apache.rocketmq.broker.controller.ReplicasManager.startBasicService(ReplicasManager.java:151)
at org.apache.rocketmq.broker.controller.ReplicasManager.lambda$start$0(ReplicasManager.java:133)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2023-03-15 16:59:14 INFO ReplicasManager_ScheduledService_2 - Update controller leader address to x.x.x.1:9878
GitHub link: https://github.com/apache/rocketmq/discussions/6354
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] weihubeats edited a discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user weihubeats edited a discussion: 自动主从切换部署不能选出master
nameserve: 2个节点
broker: 2个节点
执行
```shell
sh bin/mqadmin getBrokerEpoch -n localhost:9876 -b broker-a
```
没反应
错误log
```java
2023-03-15 16:59:04 INFO ReplicasManager_ScheduledService_1 - Update controller leader address to x.x.x.1:9878
2023-03-15 16:59:06 WARN ReplicasManager_ExecutorService_1 - Failed to start replicasManager, retry times:35, current state:FIRST_TIME_SYNC_CONTROLLER_METADATA_DONE, try it again
2023-03-15 16:59:06 ERROR ReplicasManager_ExecutorService_1 - Failed to register broker to controller
org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2012 DESC: The broker has not master, and this new registered broker can't not be elected as master
For more information, please visit the url, https://rocketmq.apache.org/docs/bestPractice/06FAQ
at org.apache.rocketmq.broker.out.BrokerOuterAPI.registerBrokerToController(BrokerOuterAPI.java:1183)
at org.apache.rocketmq.broker.controller.ReplicasManager.registerBrokerToController(ReplicasManager.java:302)
at org.apache.rocketmq.broker.controller.ReplicasManager.startBasicService(ReplicasManager.java:151)
at org.apache.rocketmq.broker.controller.ReplicasManager.lambda$start$0(ReplicasManager.java:133)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2023-03-15 16:59:11 WARN ReplicasManager_ExecutorService_1 - Failed to start replicasManager, retry times:36, current state:FIRST_TIME_SYNC_CONTROLLER_METADATA_DONE, try it again
2023-03-15 16:59:11 ERROR ReplicasManager_ExecutorService_1 - Failed to register broker to controller
org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2012 DESC: The broker has not master, and this new registered broker can't not be elected as master
For more information, please visit the url, https://rocketmq.apache.org/docs/bestPractice/06FAQ
at org.apache.rocketmq.broker.out.BrokerOuterAPI.registerBrokerToController(BrokerOuterAPI.java:1183)
at org.apache.rocketmq.broker.controller.ReplicasManager.registerBrokerToController(ReplicasManager.java:302)
at org.apache.rocketmq.broker.controller.ReplicasManager.startBasicService(ReplicasManager.java:151)
at org.apache.rocketmq.broker.controller.ReplicasManager.lambda$start$0(ReplicasManager.java:133)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2023-03-15 16:59:14 INFO ReplicasManager_ScheduledService_2 - Update controller leader address to x.x.x.1:9878
```
GitHub link: https://github.com/apache/rocketmq/discussions/6354
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] weihubeats edited a discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user weihubeats edited a discussion: 自动主从切换部署不能选出master
- rocketmq版本: 5.1.0
- nameserve: 2个节点
- broker: 2个节点
重启broker、nameserve依旧报这个错
执行
```shell
sh bin/mqadmin getBrokerEpoch -n localhost:9876 -b broker-a
```
没反应
错误log
```java
2023-03-15 16:59:04 INFO ReplicasManager_ScheduledService_1 - Update controller leader address to x.x.x.1:9878
2023-03-15 16:59:06 WARN ReplicasManager_ExecutorService_1 - Failed to start replicasManager, retry times:35, current state:FIRST_TIME_SYNC_CONTROLLER_METADATA_DONE, try it again
2023-03-15 16:59:06 ERROR ReplicasManager_ExecutorService_1 - Failed to register broker to controller
org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2012 DESC: The broker has not master, and this new registered broker can't not be elected as master
For more information, please visit the url, https://rocketmq.apache.org/docs/bestPractice/06FAQ
at org.apache.rocketmq.broker.out.BrokerOuterAPI.registerBrokerToController(BrokerOuterAPI.java:1183)
at org.apache.rocketmq.broker.controller.ReplicasManager.registerBrokerToController(ReplicasManager.java:302)
at org.apache.rocketmq.broker.controller.ReplicasManager.startBasicService(ReplicasManager.java:151)
at org.apache.rocketmq.broker.controller.ReplicasManager.lambda$start$0(ReplicasManager.java:133)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2023-03-15 16:59:11 WARN ReplicasManager_ExecutorService_1 - Failed to start replicasManager, retry times:36, current state:FIRST_TIME_SYNC_CONTROLLER_METADATA_DONE, try it again
2023-03-15 16:59:11 ERROR ReplicasManager_ExecutorService_1 - Failed to register broker to controller
org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2012 DESC: The broker has not master, and this new registered broker can't not be elected as master
For more information, please visit the url, https://rocketmq.apache.org/docs/bestPractice/06FAQ
at org.apache.rocketmq.broker.out.BrokerOuterAPI.registerBrokerToController(BrokerOuterAPI.java:1183)
at org.apache.rocketmq.broker.controller.ReplicasManager.registerBrokerToController(ReplicasManager.java:302)
at org.apache.rocketmq.broker.controller.ReplicasManager.startBasicService(ReplicasManager.java:151)
at org.apache.rocketmq.broker.controller.ReplicasManager.lambda$start$0(ReplicasManager.java:133)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2023-03-15 16:59:14 INFO ReplicasManager_ScheduledService_2 - Update controller leader address to x.x.x.1:9878
```
GitHub link: https://github.com/apache/rocketmq/discussions/6354
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] weihubeats added a comment to the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user weihubeats added a comment to the discussion: 自动主从切换部署不能选出master
添加参数 enableElectUncleanMaster = true,可以选出新的master节点。但是不明白为什么就这两个节点什么都没干回出现两个节点选不出master的情况
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5320281
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] RongtongJin added a comment to the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user RongtongJin added a comment to the discussion: 自动主从切换部署不能选出master
> > Comment options
> > cleanBrokerData 这个命令是哪个工具的，可以发个完整的吗
mqadmin工具的，最新develop分支命名为cleanBrokerMetadata
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5668050
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] weihubeats edited a discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user weihubeats edited a discussion: 自动主从切换部署不能选出master
- rocketmq版本: 5.1.0
nameserve: 2个节点
broker: 2个节点
执行
```shell
sh bin/mqadmin getBrokerEpoch -n localhost:9876 -b broker-a
```
没反应
错误log
```java
2023-03-15 16:59:04 INFO ReplicasManager_ScheduledService_1 - Update controller leader address to x.x.x.1:9878
2023-03-15 16:59:06 WARN ReplicasManager_ExecutorService_1 - Failed to start replicasManager, retry times:35, current state:FIRST_TIME_SYNC_CONTROLLER_METADATA_DONE, try it again
2023-03-15 16:59:06 ERROR ReplicasManager_ExecutorService_1 - Failed to register broker to controller
org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2012 DESC: The broker has not master, and this new registered broker can't not be elected as master
For more information, please visit the url, https://rocketmq.apache.org/docs/bestPractice/06FAQ
at org.apache.rocketmq.broker.out.BrokerOuterAPI.registerBrokerToController(BrokerOuterAPI.java:1183)
at org.apache.rocketmq.broker.controller.ReplicasManager.registerBrokerToController(ReplicasManager.java:302)
at org.apache.rocketmq.broker.controller.ReplicasManager.startBasicService(ReplicasManager.java:151)
at org.apache.rocketmq.broker.controller.ReplicasManager.lambda$start$0(ReplicasManager.java:133)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2023-03-15 16:59:11 WARN ReplicasManager_ExecutorService_1 - Failed to start replicasManager, retry times:36, current state:FIRST_TIME_SYNC_CONTROLLER_METADATA_DONE, try it again
2023-03-15 16:59:11 ERROR ReplicasManager_ExecutorService_1 - Failed to register broker to controller
org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2012 DESC: The broker has not master, and this new registered broker can't not be elected as master
For more information, please visit the url, https://rocketmq.apache.org/docs/bestPractice/06FAQ
at org.apache.rocketmq.broker.out.BrokerOuterAPI.registerBrokerToController(BrokerOuterAPI.java:1183)
at org.apache.rocketmq.broker.controller.ReplicasManager.registerBrokerToController(ReplicasManager.java:302)
at org.apache.rocketmq.broker.controller.ReplicasManager.startBasicService(ReplicasManager.java:151)
at org.apache.rocketmq.broker.controller.ReplicasManager.lambda$start$0(ReplicasManager.java:133)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2023-03-15 16:59:14 INFO ReplicasManager_ScheduledService_2 - Update controller leader address to x.x.x.1:9878
```
GitHub link: https://github.com/apache/rocketmq/discussions/6354
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] RongtongJin edited a comment on the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user RongtongJin edited a comment on the discussion: 自动主从切换部署不能选出master
@weihubeats 从你图中可以看到，当前SyncStateSet中只有一个同步副本（InSyncReplica为同步副本，NotInSyncReplica为非同步副本），因此该情况下为了保证不丢消息，只有等InSyncReplica的副本上线，才会被选为Master（除非unclean选举）。就像@TheR1sing3un所说，是否由于ip变化导致上线的两个broker均非InSyncReplica，导致无法选主，如果是的话，可以参考最新develop方案，该方案已经从Controller识别brokerIP作为broker身份改为持久化的broker编号。
此外，这种情况下，可以如下操作恢复：
（1）下线主备Broker
（2）利用cleanBrokerData命令清除Controller中该组Broker元数据
（3）删除主备Broker下的文件～/store/epochFileCheckpoint和epochFileCheckpoint.bak
（4）Broker重新上线（尽量保证旧的主备关系，先主后备上线）
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5327948
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] TheR1sing3un deleted a comment on the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user TheR1sing3un deleted a comment on the discussion: 自动主从切换部署不能选出master
> 添加参数 enableElectUncleanMaster = true,可以选出新的master节点。但是不明白为什么就这两个节点什么都没干回出现两个节点选不出master的情况
是否在重启的过程中发生ip的变化(用wsl或者容器部署会发生变化，可参考https://github.com/apache/rocketmq/issues/5745) ，因为我看这个副本组中syncStateSet中只有一个broker
<img width="530" alt="image" src="https://user-images.githubusercontent.com/87409330/225377729-3dbf2b7e-855a-4247-b3a4-33d783c6975a.png">
如果这个broker重启的时候ip变化了，重启前后这个broker就没办法对应上，所以在未开启`enableElectUncleanMaster` 的时候，就没办法在注册的时候从当前syncStateSet中选出master了, 这个目前develop分支已修复该问题，可以参考：https://github.com/apache/rocketmq/issues/5989
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5324446
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] RongtongJin edited a comment on the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user RongtongJin edited a comment on the discussion: 自动主从切换部署不能选出master
@weihubeats 当前SyncStateSet中只有一个同步副本（InSyncReplica为同步副本，NotInSyncReplica为非同步副本），因此该情况下为了保证不丢消息，只有等InSyncReplica的副本上线，才会被选为Master（除非unclean选举）。就像@TheR1sing3un所说，是否由于ip变化导致上线的两个broker均非InSyncReplica，导致无法选主。
这种情况下，可以如下操作恢复：
（1）下线主备Broker
（2）利用cleanBrokerData命令清除Controller中该组Broker元数据
（3）删除主备Broker下的文件～/store/epochFileCheckpoint和epochFileCheckpoint.bak
（4）Broker重新上线
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5327948
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] weihubeats added a comment to the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user weihubeats added a comment to the discussion: 自动主从切换部署不能选出master
应该是我本地又起来一个broker，然后本地下线，服务器上的两个broker由于落后于本地broker，所以他们都当选不了主
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5328584
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] weihubeats edited a discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user weihubeats edited a discussion: 自动主从切换部署不能选出master
nameserve: 2个节点
broker: 2个节点
错误log
```java
2023-03-15 16:59:04 INFO ReplicasManager_ScheduledService_1 - Update controller leader address to x.x.x.1:9878
2023-03-15 16:59:06 WARN ReplicasManager_ExecutorService_1 - Failed to start replicasManager, retry times:35, current state:FIRST_TIME_SYNC_CONTROLLER_METADATA_DONE, try it again
2023-03-15 16:59:06 ERROR ReplicasManager_ExecutorService_1 - Failed to register broker to controller
org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2012 DESC: The broker has not master, and this new registered broker can't not be elected as master
For more information, please visit the url, https://rocketmq.apache.org/docs/bestPractice/06FAQ
at org.apache.rocketmq.broker.out.BrokerOuterAPI.registerBrokerToController(BrokerOuterAPI.java:1183)
at org.apache.rocketmq.broker.controller.ReplicasManager.registerBrokerToController(ReplicasManager.java:302)
at org.apache.rocketmq.broker.controller.ReplicasManager.startBasicService(ReplicasManager.java:151)
at org.apache.rocketmq.broker.controller.ReplicasManager.lambda$start$0(ReplicasManager.java:133)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2023-03-15 16:59:11 WARN ReplicasManager_ExecutorService_1 - Failed to start replicasManager, retry times:36, current state:FIRST_TIME_SYNC_CONTROLLER_METADATA_DONE, try it again
2023-03-15 16:59:11 ERROR ReplicasManager_ExecutorService_1 - Failed to register broker to controller
org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2012 DESC: The broker has not master, and this new registered broker can't not be elected as master
For more information, please visit the url, https://rocketmq.apache.org/docs/bestPractice/06FAQ
at org.apache.rocketmq.broker.out.BrokerOuterAPI.registerBrokerToController(BrokerOuterAPI.java:1183)
at org.apache.rocketmq.broker.controller.ReplicasManager.registerBrokerToController(ReplicasManager.java:302)
at org.apache.rocketmq.broker.controller.ReplicasManager.startBasicService(ReplicasManager.java:151)
at org.apache.rocketmq.broker.controller.ReplicasManager.lambda$start$0(ReplicasManager.java:133)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2023-03-15 16:59:14 INFO ReplicasManager_ScheduledService_2 - Update controller leader address to x.x.x.1:9878
```
GitHub link: https://github.com/apache/rocketmq/discussions/6354
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] RongtongJin edited a comment on the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user RongtongJin edited a comment on the discussion: 自动主从切换部署不能选出master
@weihubeats 从你图中可以看到，当前SyncStateSet中只有一个同步副本（InSyncReplica为同步副本，NotInSyncReplica为非同步副本），因此该情况下为了保证不丢消息，只有等InSyncReplica的副本上线，才会被选为Master（除非unclean选举）。就像@TheR1sing3un所说，是否由于ip变化导致上线的两个broker均非InSyncReplica，导致无法选主，如果
此外，这种情况下，可以如下操作恢复：
（1）下线主备Broker
（2）利用cleanBrokerData命令清除Controller中该组Broker元数据
（3）删除主备Broker下的文件～/store/epochFileCheckpoint和epochFileCheckpoint.bak
（4）Broker重新上线（尽量保证旧的主备关系上线）
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5327948
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] RongtongJin added a comment to the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user RongtongJin added a comment to the discussion: 自动主从切换部署不能选出master
@weihubeats 当前SyncStateSet中只有一个同步副本（InSyncReplica为同步副本，NotInSyncReplica为非同步副本），因此该情况下为了保证不丢消息，只有等InSyncReplica的副本上线，才会被选为Master（除非unclean选举）。就像@TheR1sing3un所说，是否由于ip变化导致上线的两个broker均非InSyncReplica，导致无法选主。
这种情况下，可以如下操作恢复：
（1）下线主备Broker
（2）利用cleanBrokerData命令清除Controller中该组Broker元数据
（3）删除Broker下～/store/epochFileCheckpoint和epochFileCheckpoint.bak
（4）Broker重新上线
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5327948
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] weihubeats added a comment to the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user weihubeats added a comment to the discussion: 自动主从切换部署不能选出master
没有使用wsl和容器部署，所以机器ip应该是不会变的
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5328555
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] zuoHx580 added a comment to the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user zuoHx580 added a comment to the discussion: 自动主从切换部署不能选出master
> Comment options
cleanBrokerData 这个命令是哪个工具的，可以发个完整的吗
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5658530
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] RongtongJin edited a comment on the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user RongtongJin edited a comment on the discussion: 自动主从切换部署不能选出master
@weihubeats 从你图中可以看到，当前SyncStateSet中只有一个同步副本（InSyncReplica为同步副本，NotInSyncReplica为非同步副本），因此该情况下为了保证不丢消息，只有等InSyncReplica的副本上线，才会被选为Master（除非unclean选举）。就像@TheR1sing3un所说，是否由于ip变化导致上线的两个broker均非InSyncReplica，导致无法选主，如果是的话，可以参考@TheR1sing3un解决方法（已经合入develop分支），该方案已经从Controller识别brokerIP作为broker身份改为持久化的broker编号。
此外，这种情况下，可以如下操作恢复：
（1）下线主备Broker
（2）利用cleanBrokerData命令清除Controller中该组Broker元数据
（3）删除主备Broker下的文件～/store/epochFileCheckpoint和epochFileCheckpoint.bak
（4）Broker重新上线（尽量保证旧的主备关系，先主后备上线）
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5327948
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] RongtongJin edited a comment on the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user RongtongJin edited a comment on the discussion: 自动主从切换部署不能选出master
@weihubeats 从你图中可以看到，当前SyncStateSet中只有一个同步副本（InSyncReplica为同步副本，NotInSyncReplica为非同步副本），因此该情况下为了保证不丢消息，只有等InSyncReplica的副本上线，才会被选为Master（除非unclean选举）。就像@TheR1sing3un所说，是否由于ip变化导致上线的两个broker均非InSyncReplica，导致无法选主。
这种情况下，可以如下操作恢复：
（1）下线主备Broker
（2）利用cleanBrokerData命令清除Controller中该组Broker元数据
（3）删除主备Broker下的文件～/store/epochFileCheckpoint和epochFileCheckpoint.bak
（4）Broker重新上线
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5327948
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] weihubeats added a comment to the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user weihubeats added a comment to the discussion: 自动主从切换部署不能选出master
好的，非常感觉，我好像明白了，因为为了查看原因又在本地启了一个broker，导致本地的broker的数据文件是最新的，所以服务器上的两个broker都当不了master
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5328573
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] zuoHx580 added a comment to the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user zuoHx580 added a comment to the discussion: 自动主从切换部署不能选出master
> > > Comment options
> > > cleanBrokerData 这个命令是哪个工具的，可以发个完整的吗
>
> mqadmin工具的，最新develop分支命名为cleanBrokerMetadata
好的，非常感谢
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5668073
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] RongtongJin edited a comment on the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user RongtongJin edited a comment on the discussion: 自动主从切换部署不能选出master
@weihubeats 从你图中可以看到，当前SyncStateSet中只有一个同步副本（InSyncReplica为同步副本，NotInSyncReplica为非同步副本），因此该情况下为了保证不丢消息，只有等InSyncReplica的副本上线，才会被选为Master（除非unclean选举）。就像@TheR1sing3un所说，是否由于ip变化导致上线的两个broker均非InSyncReplica，导致无法选主，如果
此外，这种情况下，可以如下操作恢复：
（1）下线主备Broker
（2）利用cleanBrokerData命令清除Controller中该组Broker元数据
（3）删除主备Broker下的文件～/store/epochFileCheckpoint和epochFileCheckpoint.bak
（4）Broker重新上线（尽量保证旧的主备关系，先主后备上线）
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5327948
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] TheR1sing3un added a comment to the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user TheR1sing3un added a comment to the discussion: 自动主从切换部署不能选出master
> 添加参数 enableElectUncleanMaster = true,可以选出新的master节点。但是不明白为什么就这两个节点什么都没干回出现两个节点选不出master的情况
是否在重启的过程中发生ip的变化(用wsl或者容器部署会发生变化，可参考https://github.com/apache/rocketmq/issues/5745) ，因为我看这个副本组中syncStateSet中只有一个broker
<img width="530" alt="image" src="https://user-images.githubusercontent.com/87409330/225377729-3dbf2b7e-855a-4247-b3a4-33d783c6975a.png">
如果这个broker重启的时候ip变化了，重启前后这个broker就没办法对应上，所以在未开启`enableElectUncleanMaster` 的时候，就没办法在注册的时候从当前syncStateSet中选出master了, 这个目前develop分支已修复该问题，可以参考：https://github.com/apache/rocketmq/issues/5989
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5324446
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org
[GitHub] [rocketmq] TheR1sing3un added a comment to the discussion: 自动主从切换部署不能选出master
Posted by GitBox <gi...@apache.org>.
GitHub user TheR1sing3un added a comment to the discussion: 自动主从切换部署不能选出master
是否在重启的过程中发生ip的变化(用wsl或者容器部署会发生变化，可参考https://github.com/apache/rocketmq/issues/5745) ，因为我看这个副本组中syncStateSet中只有一个broker
<img width="530" alt="image" src="https://user-images.githubusercontent.com/87409330/225377729-3dbf2b7e-855a-4247-b3a4-33d783c6975a.png">
如果这个broker重启的时候ip变化了，重启前后这个broker就没办法对应上，所以在未开启`enableElectUncleanMaster` 的时候，就没办法在注册的时候从当前syncStateSet中选出master了, 这个目前develop分支已修复该问题，可以参考：https://github.com/apache/rocketmq/issues/5989
GitHub link: https://github.com/apache/rocketmq/discussions/6354#discussioncomment-5324456
----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org