You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2020/09/07 11:11:26 UTC
[GitHub] [rocketmq] nervose opened a new issue #2291: one slave could not synchronize data in a 2-master-2-slave-sync-cluster
nervose opened a new issue #2291:
URL: https://github.com/apache/rocketmq/issues/2291
The issue tracker is **ONLY** used for bug report(feature request need to follow [RIP process](https://github.com/apache/rocketmq/wiki/RocketMQ-Improvement-Proposal)). Keep in mind, please check whether there is an existing same report before your raise a new one.
Alternately (especially if your communication is not a bug report), you can send mail to our [mailing lists](http://rocketmq.apache.org/about/contact/). We welcome any friendly suggestions, bug fixes, collaboration and other improvements.
Please ensure that your bug report is clear and that it is complete. Otherwise, we may be unable to understand it or to reproduce it, either of which would prevent us from fixing the bug. We strongly recommend the report(bug report or feature request) could include some hints as the following:
**BUG REPORT**
1. Please describe the issue you observed:
one slave (broker-slave-b) do not synchronize data in a 2-master-2-slave-sync-cluster(NOT CREATE commitlog, consumequeue, index under log dir )
- What did you see instead?
After send serval messages , about half of resut is SLAVE_NOT_AVAILABLE
2. Please tell us about your environment:
broker-master-a.conf:
#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字 broker-a、broker-b
brokerName=broker-a
#0 表示 Master，>0 表示 Slave
brokerId=0
#nameServer地址，分号分割
namesrvAddr=namesrv-a:9876;namesrv-b:9876
#当前broker监听的IP,此处为宿主机地址
brokerIP1=192.168.75.140
#存在broker主从时，在broker主节点上配置了brokerIP2的话,broker从节点会连接主节点配置的brokerIP2来同步
brokerIP2=broker-master-a
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口
listenPort=10911
#删除文件时间点，默认凌晨 4点
deleteWhen=04
#文件保留时间，默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/opt/store
#commitLog 存储路径
storePathCommitLog=/opt/store/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/opt/store/consumequeue
#消息索引存储路径
storePathIndex=/opt/store/index
#checkpoint 文件存储路径
storeCheckpoint=/opt/store/checkpoint
#abort 文件存储路径
abortFile=/opt/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=SYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
broker-master-b.conf
#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字 broker-a、broker-b
brokerName=broker-b
#0 表示 Master，>0 表示 Slave
brokerId=0
#nameServer地址，分号分割
namesrvAddr=namesrv-a:9876;namesrv-b:9876
#当前broker监听的IP,此处为宿主机地址
brokerIP1=192.168.75.140
#存在broker主从时，在broker主节点上配置了brokerIP2的话,broker从节点会连接主节点配置的brokerIP2来同步
brokerIP2=broker-master-a
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口
listenPort=11911
#删除文件时间点，默认凌晨 4点
deleteWhen=04
#文件保留时间，默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/opt/store
#commitLog 存储路径
storePathCommitLog=/opt/store/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/opt/store/consumequeue
#消息索引存储路径
storePathIndex=/opt/store/index
#checkpoint 文件存储路径
storeCheckpoint=/opt/store/checkpoint
#abort 文件存储路径
abortFile=/opt/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=SYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
broker-slave-a.conf
#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字 broker-a、broker-b
brokerName=broker-b
#0 表示 Master，>0 表示 Slave
brokerId=0
#nameServer地址，分号分割
namesrvAddr=namesrv-a:9876;namesrv-b:9876
#当前broker监听的IP,此处为宿主机地址
brokerIP1=192.168.75.140
#存在broker主从时，在broker主节点上配置了brokerIP2的话,broker从节点会连接主节点配置的brokerIP2来同步
brokerIP2=broker-master-a
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口
listenPort=11911
#删除文件时间点，默认凌晨 4点
deleteWhen=04
#文件保留时间，默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/opt/store
#commitLog 存储路径
storePathCommitLog=/opt/store/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/opt/store/consumequeue
#消息索引存储路径
storePathIndex=/opt/store/index
#checkpoint 文件存储路径
storeCheckpoint=/opt/store/checkpoint
#abort 文件存储路径
abortFile=/opt/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=SYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
broker-slave-b.conf
#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字 broker-a、broker-b
brokerName=broker-b
#0 表示 Master，>0 表示 Slave
brokerId=0
#nameServer地址，分号分割
namesrvAddr=namesrv-a:9876;namesrv-b:9876
#当前broker监听的IP,此处为宿主机地址
brokerIP1=192.168.75.140
#存在broker主从时，在broker主节点上配置了brokerIP2的话,broker从节点会连接主节点配置的brokerIP2来同步
brokerIP2=broker-master-a
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口
listenPort=11911
#删除文件时间点，默认凌晨 4点
deleteWhen=04
#文件保留时间，默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/opt/store
#commitLog 存储路径
storePathCommitLog=/opt/store/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/opt/store/consumequeue
#消息索引存储路径
storePathIndex=/opt/store/index
#checkpoint 文件存储路径
storeCheckpoint=/opt/store/checkpoint
#abort 文件存储路径
abortFile=/opt/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=SYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
docker-compose.yaml
version: '3.5'
services:
namesrv-a:
image: foxiswho/rocketmq:4.7.0
container_name: namesrv-a
ports:
- 9876:9876
volumes:
- /data/rocketmq/nameserver-a/logs:/opt/logs
- /data/rocketmq/nameserver-a/store:/opt/store
environment:
JAVA_OPT_EXT: "-Duser.home=/opt -Xms512M -Xmx512M -Xmn128m"
command: ["sh","mqnamesrv"]
namesrv-b:
image: foxiswho/rocketmq:4.7.0
container_name: namesrv-b
ports:
- 9877:9876
volumes:
- /data/rocketmq/nameserver-b/logs:/opt/logs
- /data/rocketmq/nameserver-b/store:/opt/store
environment:
JAVA_OPT_EXT: "-Duser.home=/opt -Xms512M -Xmx512M -Xmn128m"
command: ["sh","mqnamesrv"]
broker-master-a:
image: foxiswho/rocketmq:4.7.0
container_name: broker-master-a
ports:
- 10909:10909
- 10911:10911
- 10912:10912
volumes:
- /data/rocketmq/broker-master-a/logs:/opt/logs
- /data/rocketmq/broker-master-a/store:/opt/store
- /data/rocketmq/broker-master-a/brokerconf/broker.conf:/etc/rocketmq/broker.conf
environment:
JAVA_OPT_EXT: "-Duser.home=/opt -Xms512M -Xmx512M -Xmn128m"
command: ["sh","mqbroker","-c","/etc/rocketmq/broker.conf","-n","namesrv-a:9876;namesrv-b:9876","autoCreateTopicEnable=true"]
depends_on:
- namesrv-a
- namesrv-b
broker-master-b:
image: foxiswho/rocketmq:4.7.0
container_name: broker-master-b
ports:
- 11909:11909
- 11911:11911
- 11912:11912
volumes:
- /data/rocketmq/broker-master-b/logs:/opt/logs
- /data/rocketmq/broker-master-b/store:/opt/store
- /data/rocketmq/broker-master-b/brokerconf/broker.conf:/etc/rocketmq/broker.conf
environment:
JAVA_OPT_EXT: "-Duser.home=/opt -Xms512M -Xmx512M -Xmn128m"
command: ["sh","mqbroker","-c","/etc/rocketmq/broker.conf","-n","namesrv-a:9876;namesrv-b:9876","autoCreateTopicEnable=true"]
depends_on:
- namesrv-a
- namesrv-b
broker-slave-a:
image: foxiswho/rocketmq:4.7.0
container_name: broker-slave-a
ports:
- 12909:12909
- 12911:12911
- 12912:12912
volumes:
- /data/rocketmq/broker-slave-a/logs:/opt/logs
- /data/rocketmq/broker-slave-a/store:/opt/store
- /data/rocketmq/broker-slave-a/brokerconf/broker.conf:/etc/rocketmq/broker.conf
environment:
JAVA_OPT_EXT: "-Duser.home=/opt -Xms512M -Xmx512M -Xmn128m"
command: ["sh","mqbroker","-c","/etc/rocketmq/broker.conf","-n","namesrv-a:9876;namesrv-b:9876","autoCreateTopicEnable=true"]
depends_on:
- namesrv-a
- namesrv-b
broker-slave-b:
image: foxiswho/rocketmq:4.7.0
container_name: broker-slave-b
ports:
- 13909:13909
- 13911:13911
- 13912:13912
volumes:
- /data/rocketmq/broker-slave-b/logs:/opt/logs
- /data/rocketmq/broker-slave-b/store:/opt/store
- /data/rocketmq/broker-slave-b/brokerconf/broker.conf:/etc/rocketmq/broker.conf
environment:
JAVA_OPT_EXT: "-Duser.home=/opt -Xms512M -Xmx512M -Xmn128m"
command: ["sh","mqbroker","-c","/etc/rocketmq/broker.conf","-n","namesrv-a:9876;namesrv-b:9876","autoCreateTopicEnable=true"]
depends_on:
- namesrv-a
- namesrv-b
rmqconsole:
image: styletang/rocketmq-console-ng
container_name: rmqconsole
ports:
- 8180:8080
environment:
JAVA_OPTS: "-Drocketmq.namesrv.addr=namesrv-a:9876;namesrv-b:9876 -Dcom.rocketmq.sendMessageWithVIPChannel=false"
depends_on:
- namesrv-a
- namesrv-b
3. Other information (e.g. detailed explanation, logs, related issues, suggestions how to fix, etc):
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] nervose closed issue #2291: one slave could not synchronize data in a 2-master-2-slave-sync-cluster
Posted by GitBox <gi...@apache.org>.
nervose closed issue #2291:
URL: https://github.com/apache/rocketmq/issues/2291
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org