You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "john1337 (via GitHub)" <gi...@apache.org> on 2025/11/10 07:43:16 UTC
[I] [Enhancement] rocketmq 单节点发送消息tps只能到1万多 [rocketmq]
john1337 opened a new issue, #9829:
URL: https://github.com/apache/rocketmq/issues/9829
### Before Creating the Enhancement Request
- [x] I have confirmed that this should be classified as an enhancement rather than a bug/feature.
### Summary
使用rocketmq官方的benchmak producer.sh脚本测试rocketmq发送性能，用的rocketmq 5.3.3版本
部署了1个broker节点，
验证脚本如下：./producer.sh -t cluster-perf-tst8 -w 20 -s 1024 -n rocketmq2-nameserver.rocketmq:9876
性能表现比较差，发送消息tps只能到1万多，broker给的堆内存为8G，异步落盘，使用的也是ssd硬盘，不知道为什么性能这么差
### Motivation
broker性能比较差
### Describe the Solution You'd Like
不知哪里配置不够，但是性能着实比较差
### Describe Alternatives You've Considered
希望提升rocketmq tps性能
### Additional Context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Enhancement] rocketmq 单节点发送消息tps只能到1万多 [rocketmq]
Posted by "huangjuncong (via GitHub)" <gi...@apache.org>.
huangjuncong commented on issue #9829:
URL: https://github.com/apache/rocketmq/issues/9829#issuecomment-3521764981
巧了，我也正在部署5.3.3集群，今天刚压测的，broker 一个节点,jvm 给8G，TPS打到3.8万，采用了一半CPU 和一半内存。性能瓶颈在硬盘，util飙到95%
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Enhancement] rocketmq 单节点发送消息tps只能到1万多 [rocketmq]
Posted by "lizhimins (via GitHub)" <gi...@apache.org>.
lizhimins commented on issue #9829:
URL: https://github.com/apache/rocketmq/issues/9829#issuecomment-3673747313
使用 5.3.4 尝试一下，另外可以看下 dstat 和修改一下 brokerConfig 的线程数
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Enhancement] rocketmq 单节点发送消息tps只能到1万多 [rocketmq]
Posted by "john1337 (via GitHub)" <gi...@apache.org>.
john1337 commented on issue #9829:
URL: https://github.com/apache/rocketmq/issues/9829#issuecomment-3524471932
> 巧了，我也正在部署5.3.3集群，今天刚压测的，broker 一个节点,jvm 给8G，TPS打到3.8万，采用了一半CPU 和一半内存。性能瓶颈在硬盘，util飙到95%
请问你的验证脚本是啥，还有就是proxy、broker的参数
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Enhancement] rocketmq 单节点发送消息tps只能到1万多 [rocketmq]
Posted by "huangjuncong (via GitHub)" <gi...@apache.org>.
huangjuncong commented on issue #9829:
URL: https://github.com/apache/rocketmq/issues/9829#issuecomment-3532331403
> > > > 巧了，我也正在部署5.3.3集群，今天刚压测的，broker 一个节点,jvm 给8G，TPS打到3.8万，采用了一半CPU 和一半内存。性能瓶颈在硬盘，util飙到95%
> > >
> > >
> > > 请问你的验证脚本是啥，还有就是proxy、broker的参数
> > > java -cp 'lib/*' org.apache.rocketmq.example.benchmark.Producer -n "mq1:9876;mq2:9876;mq3:9876" -t "normalTopic" -s 1024 -w 150
>
> 咱俩的写入线程数是不一样的，请问你这边有优化过broker参数么（比如broker的sendmessage线程数等等），还是直接部署的，不知你那边验证的延迟有多大，这个也很重要
我是docker compose 部署的，
`version: '3.8'
services:
rmqnamesrv:
image: apache/rocketmq:5.3.3
container_name: rmqnamesrv
ports:
- "9876:9876"
volumes:
# 将日志挂载到 /data
- /data/rocketmq/namesrv/logs:/home/rocketmq/logs
# GC 日志挂载
- /data/rocketmq/namesrv/gclogs:/home/rocketmq/gclogs
networks:
- rocketmq
extra_hosts:
- "mq1:xxx.xx.xx.xx" # 示例：mq1 映射到 xxx.xx.xx.xx
- "mq2:xxx.xx.xx.xx" # 示例：mq2 映射到 xxx.xx.xx.xx
- "mq3:xxx.xx.xx.xx" # 示例：mq3 映射到 xxx.xx.xx.xx
environment:
- JAVA_OPT_EXT=-Xms512m -Xmx512m -Xmn256m -XX:MetaspaceSize=64m -XX:MaxMetaspaceSize=128m -XX:MaxGCPauseMillis=100 -XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=100M -Xloggc:/home/rocketmq/gclogs/gc.log
command: sh mqnamesrv
restart: unless-stopped
user: "3000:3000"
deploy:
resources:
limits:
cpus: '0.5'
memory: 1G
reservations:
cpus: '0.1'
memory: 256M
rmqbroker-c:
image: apache/rocketmq:5.3.3
container_name: rmqbroker-c
ports:
- "10909:10909"
- "10911:10911"
- "10912:10912"
- "8080:8080"
- "8081:8081"
extra_hosts:
- "mq1:xxx.xx.xx.xx" # 示例：mq1 映射到 xxx.xx.xx.xx
- "mq2:xxx.xx.xx.xx" # 示例：mq2 映射到 xxx.xx.xx.xx
- "mq3:xxx.xx.xx.xx" # 示例：mq3 映射到 xxx.xx.xx.xx
environment:
- NAMESRV_ADDR=mq1:9876;mq2:9876;mq3:9876
- JAVA_OPT_EXT=-Xms8g -Xmx8g -Xmn4g -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m -XX:MaxGCPauseMillis=200 -XX:MaxDirectMemorySize=2g -XX:G1HeapRegionSize=16m -XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Xloggc:/home/rocketmq/gclogs/gc.log
depends_on:
- rmqnamesrv
volumes:
# 配置文件从 /opt/rocketmq 加载
- ./broker-c/conf/broker.conf:/home/rocketmq/rocketmq-5.3.3/conf/broker.conf
# 将日志和存储挂载到 /data
- /data/rocketmq/broker-c/logs:/home/rocketmq/logs
- /data/rocketmq/broker-c/store:/home/rocketmq/store
# GC 日志挂载
- /data/rocketmq/broker-c/gclogs:/home/rocketmq/gclogs
networks:
- rocketmq
command: sh mqbroker -c /home/rocketmq/rocketmq-5.3.3/conf/broker.conf --enable-proxy
restart: unless-stopped
user: "3000:3000"
deploy:
resources:
limits:
cpus: '4.0'
memory: 12G
reservations:
cpus: '1.0'
memory: 3G
rmqbroker-a-s:
image: apache/rocketmq:5.3.3
container_name: rmqbroker-a-s
ports:
- "11909:11909" # Slave A需要不同端口
- "11911:11911"
- "11912:11912"
extra_hosts:
- "mq1:xxx.xx.xx.xx" # 示例：mq1 映射到 xxx.xx.xx.xx
- "mq2:xxx.xx.xx.xx" # 示例：mq2 映射到 xxx.xx.xx.xx
- "mq3:xxx.xx.xx.xx" # 示例：mq3 映射到 xxx.xx.xx.xx
environment:
- NAMESRV_ADDR=mq1:9876;mq2:9876;mq3:9876
- JAVA_OPT_EXT=-Xms4g -Xmx4g -Xmn2g -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=256m -XX:MaxGCPauseMillis=200 -XX:MaxDirectMemorySize=1g -XX:G1HeapRegionSize=16m -XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=100M -Xloggc:/home/rocketmq/gclogs/gc.log
depends_on:
- rmqnamesrv
volumes:
# 配置文件从 /opt/rocketmq 加载
- ./broker-a-s/conf/broker.conf:/home/rocketmq/rocketmq-5.3.3/conf/broker.conf
# 将日志和存储挂载到 /data
- /data/rocketmq/broker-a-s/logs:/home/rocketmq/logs
- /data/rocketmq/broker-a-s/store:/home/rocketmq/store
# GC 日志挂载
- /data/rocketmq/broker-a-s/gclogs:/home/rocketmq/gclogs
networks:
- rocketmq
command: sh mqbroker -c /home/rocketmq/rocketmq-5.3.3/conf/broker.conf
restart: unless-stopped
user: "3000:3000"
deploy:
resources:
limits:
cpus: '2.5'
memory: 8G
reservations:
cpus: '0.75'
memory: 2G
networks:
rocketmq:
name: rocketmq
driver: bridge`
<img width="1293" height="471" alt="Image" src="https://github.com/user-attachments/assets/b1f5ab3d-3bd6-44a3-93c1-39f78b7c1466" />
<img width="1146" height="757" alt="Image" src="https://github.com/user-attachments/assets/01cc8632-25d5-468a-b612-d0bf515875c1" />
%util 打到95% 性能瓶颈在硬盘，CPU和内存才用到一半，我是在8核32G机器测的。内存和CPU 不是瓶颈因素
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Enhancement] rocketmq 单节点发送消息tps只能到1万多 [rocketmq]
Posted by "john1337 (via GitHub)" <gi...@apache.org>.
john1337 commented on issue #9829:
URL: https://github.com/apache/rocketmq/issues/9829#issuecomment-3509950251
./producer.sh -t cluster-perf-tst8 -w 1-s 1024 -n rocketmq2-nameserver.rocketmq:9876 测试tps在700-1000之间
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Enhancement] rocketmq 单节点发送消息tps只能到1万多 [rocketmq]
Posted by "john1337 (via GitHub)" <gi...@apache.org>.
john1337 commented on issue #9829:
URL: https://github.com/apache/rocketmq/issues/9829#issuecomment-3524872165
> > > 巧了，我也正在部署5.3.3集群，今天刚压测的，broker 一个节点,jvm 给8G，TPS打到3.8万，采用了一半CPU 和一半内存。性能瓶颈在硬盘，util飙到95%
> >
> >
> > 请问你的验证脚本是啥，还有就是proxy、broker的参数
> > java -cp 'lib/*' org.apache.rocketmq.example.benchmark.Producer -n "mq1:9876;mq2:9876;mq3:9876" -t "normalTopic" -s 1024 -w 150
咱俩的写入线程数是不一样的，请问你这边有优化过broker参数么（比如broker的sendmessage线程数等等），还是直接部署的，不知你那边验证的延迟有多大，这个也很重要
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Enhancement] rocketmq 单节点发送消息tps只能到1万多 [rocketmq]
Posted by "huangjuncong (via GitHub)" <gi...@apache.org>.
huangjuncong commented on issue #9829:
URL: https://github.com/apache/rocketmq/issues/9829#issuecomment-3524828486
> > 巧了，我也正在部署5.3.3集群，今天刚压测的，broker 一个节点,jvm 给8G，TPS打到3.8万，采用了一半CPU 和一半内存。性能瓶颈在硬盘，util飙到95%
>
> 请问你的验证脚本是啥，还有就是proxy、broker的参数
java -cp 'lib/*' org.apache.rocketmq.example.benchmark.Producer -n "mq1:9876;mq2:9876;mq3:9876" -t "normalTopic" -s 1024 -w 150
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org