You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2022/02/08 07:37:52 UTC
[GitHub] [rocketmq] areyouok opened a new issue #3825: The fie
areyouok opened a new issue #3825:
URL: https://github.com/apache/rocketmq/issues/3825
The issue tracker is **ONLY** used for bug report(feature request need to follow [RIP process](https://github.com/apache/rocketmq/wiki/RocketMQ-Improvement-Proposal)). Keep in mind, please check whether there is an existing same report before your raise a new one.
Alternately (especially if your communication is not a bug report), you can send mail to our [mailing lists](http://rocketmq.apache.org/about/contact/). We welcome any friendly suggestions, bug fixes, collaboration and other improvements.
Please ensure that your bug report is clear and that it is complete. Otherwise, we may be unable to understand it or to reproduce it, either of which would prevent us from fixing the bug. We strongly recommend the report(bug report or feature request) could include some hints as the following:
**BUG REPORT**
1. Please describe the issue you observed:
- What did you do (The steps to reproduce)?
- What did you expect to see?
- What did you see instead?
2. Please tell us about your environment:
3. Other information (e.g. detailed explanation, logs, related issues, suggestions how to fix, etc):
**FEATURE REQUEST**
1. Please describe the feature you are requesting.
2. Provide any additional detail on your proposed use case for this feature.
2. Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?
4. If there are some sub-tasks using -[] for each subtask and create a corresponding issue to map to the sub task:
- [sub-task1-issue-number](example_sub_issue1_link_here): sub-task1 description here,
- [sub-task2-issue-number](example_sub_issue2_link_here): sub-task2 description here,
- ...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] areyouok closed issue #3825: The TCP SO_SNDBUF/SO_RCVBUF is fixed and too small, decrease throughput greatly
Posted by GitBox <gi...@apache.org>.
areyouok closed issue #3825:
URL: https://github.com/apache/rocketmq/issues/3825
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] areyouok edited a comment on issue #3825: The TCP SO_SNDBUF/SO_RCVBUF is fixed and too small, decrease throughput greatly
Posted by GitBox <gi...@apache.org>.
areyouok edited a comment on issue #3825:
URL: https://github.com/apache/rocketmq/issues/3825#issuecomment-1032385516
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] areyouok commented on issue #3825: The TCP SO_SNDBUF/SO_RCVBUF is fixed and too small, decrease throughput greatly
Posted by GitBox <gi...@apache.org>.
areyouok commented on issue #3825:
URL: https://github.com/apache/rocketmq/issues/3825#issuecomment-1032385516
Cluster produce/consume test
* 6 producer and 6 consumer in 6 machines, linux kernel 3.x
* 4 SyncMaster and 4 Slave in 2 machines, linux kernel 4.x
* each broker has 18 queues for test topic
* each producer has 400 threads
* network latency: 1.61ms
although the old version has poor performance in single tcp connection, but there is 6*4 tcp connection for produce and consume. So the produce performance of old version is not too bad, but it's consume qps can't keep up with the produce tps.
version | body size | producer tps | consumer qps
-- | -- | -- | --
Old | 250 | 18w*4 | 13w*4
Old | 4,000 | 7.2w*4 | 1.8w*4
Old | 16,000 | 1.9w*4 | 4.6k*4
Old | 100,000 | 3000*4 | 800*4
New | 250 | 17.5w*4 | 17.5w*4
New | 4,000 | 6.8w*4 | 6.8w*4
New | 16,000 | 1.8w*4 | 1.8w*4
New | 100,000 | 2800*4 | 2800*4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] areyouok commented on issue #3825: The TCP SO_SNDBUF/SO_RCVBUF is fixed and too small, decrease throughput greatly
Posted by GitBox <gi...@apache.org>.
areyouok commented on issue #3825:
URL: https://github.com/apache/rocketmq/issues/3825#issuecomment-1032329565
Here is an article about this problem (chinese version):
https://www.infoq.cn/article/sfjkj1c5bz2koxsxybho
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] areyouok edited a comment on issue #3825: The TCP SO_SNDBUF/SO_RCVBUF is fixed and too small, decrease throughput greatly
Posted by GitBox <gi...@apache.org>.
areyouok edited a comment on issue #3825:
URL: https://github.com/apache/rocketmq/issues/3825#issuecomment-1032385516
Cluster produce/consume test
* 6 producer and 6 consumer in 6 machines, linux kernel 3.x
* 4 SyncMaster and 4 Slave in 2 machines, linux kernel 4.x
* each broker has 18 queues for test topic
* each producer has 400 threads
* network latency: 1.61ms
although the old version has poor performance in single tcp connection, but there is 6*4 tcp connection for produce and consume. So the produce performance of old version is not too bad, but it's consume qps can't keep up with the produce tps.
version | body size | producer tps | consumer qps
-- | -- | -- | --
Old | 250 | 180,000*4 | 130,000*4
Old | 4,000 | 72,000*4 | 18,000*4
Old | 16,000 | 19,000*4 | 4,600*4
Old | 100,000 | 3000*4 | 800*4
New | 250 | 175,000*4 | 175,000*4
New | 4,000 | 68,000*4 | 68,000*4
New | 16,000 | 18,000*4 | 180,000*4
New | 100,000 | 2800*4 | 2800*4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] areyouok edited a comment on issue #3825: The TCP SO_SNDBUF/SO_RCVBUF is fixed and too small, decrease throughput greatly
Posted by GitBox <gi...@apache.org>.
areyouok edited a comment on issue #3825:
URL: https://github.com/apache/rocketmq/issues/3825#issuecomment-1032385516
Cluster produce/consume test
* 6 producer and 6 consumer in 6 machines, linux kernel 3.x
* 4 SyncMaster and 4 Slave in 2 machines, linux kernel 4.x
* each broker has 18 queues for test topic
* each producer has 400 threads, sync send
* network latency: 1.61ms
although the old version has poor performance in single tcp connection, but there are 6*4 tcp connections for produce and consume, respectively. So the produce performance of old version is not too bad, but it's consume qps can't keep up with the produce tps.
version | message body size | producer tps | consumer qps
-- | -- | -- | --
Old | 250 | 180,000*4 | 130,000*4
Old | 4,000 | 72,000*4 | 18,000*4
Old | 16,000 | 19,000*4 | 4,600*4
Old | 100,000 | 3000*4 | 800*4
New | 250 | 175,000*4 | 175,000*4
New | 4,000 | 68,000*4 | 68,000*4
New | 16,000 | 18,000*4 | 180,000*4
New | 100,000 | 2800*4 | 2800*4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] areyouok commented on issue #3825: The TCP SO_SNDBUF/SO_RCVBUF is fixed and too small, decrease throughput greatly
Posted by GitBox <gi...@apache.org>.
areyouok commented on issue #3825:
URL: https://github.com/apache/rocketmq/issues/3825#issuecomment-1032397075
Single producer/broker test, no consumer
* 1 producer , linux kernel 3.x
* 1 SyncMaster 1 Slave, linux kernel 4.x
* 18 queues for test topic
* 400 threads
* network latency: 1.61ms
version | body size | produce tps
-- | -- | --
Old | 250 | 88,000
Old | 4,000 | 18,200
Old | 16,000 | 4,860
Old | 100,000 | 750
New | 250 | 82,000
New | 4,000 | 70,000
New | 16,000 | 29,500
New | 100,000 | 5,950
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] areyouok edited a comment on issue #3825: The TCP SO_SNDBUF/SO_RCVBUF is fixed and too small, decrease throughput greatly
Posted by GitBox <gi...@apache.org>.
areyouok edited a comment on issue #3825:
URL: https://github.com/apache/rocketmq/issues/3825#issuecomment-1032385516
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] areyouok edited a comment on issue #3825: The TCP SO_SNDBUF/SO_RCVBUF is fixed and too small, decrease throughput greatly
Posted by GitBox <gi...@apache.org>.
areyouok edited a comment on issue #3825:
URL: https://github.com/apache/rocketmq/issues/3825#issuecomment-1032385516
Cluster produce/consume test
* 6 producer and 6 consumer in 6 machines, linux kernel 3.x
* 4 SyncMaster and 4 Slave in 2 machines, linux kernel 4.x
* each broker has 18 queues for test topic
* each producer has 400 threads, sync send
* network latency: 1.61ms
although the old version has poor performance in single tcp connection, but there are 6*4 tcp connections for produce and consume, respectively. So the produce performance of old version is not too bad, but it's consume qps can't keep up with the produce tps.
version | message body size | producer tps | consumer qps
-- | -- | -- | --
Old | 250 | 180,000*4 | 130,000*4
Old | 4,000 | 72,000*4 | 18,000*4
Old | 16,000 | 19,000*4 | 4,600*4
Old | 100,000 | 3000*4 | 800*4
New | 250 | 175,000*4 | 175,000*4
New | 4,000 | 68,000*4 | 68,000*4
New | 16,000 | 18,000*4 | 18,000*4
New | 100,000 | 2800*4 | 2800*4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] dongeforever commented on issue #3825: The TCP SO_SNDBUF/SO_RCVBUF is fixed and too small, decrease throughput greatly
Posted by GitBox <gi...@apache.org>.
dongeforever commented on issue #3825:
URL: https://github.com/apache/rocketmq/issues/3825#issuecomment-1032400869
Great Catch!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] areyouok commented on issue #3825: The TCP SO_SNDBUF/SO_RCVBUF is fixed and too small, decrease throughput greatly
Posted by GitBox <gi...@apache.org>.
areyouok commented on issue #3825:
URL: https://github.com/apache/rocketmq/issues/3825#issuecomment-1032373380
in fact we have more data.
* A. remove SO_SNDBUF/SO_RCVBUF/WRITE_BUFFER_WATER_MARK setting
* B. set SO_SNDBUF/SO_RCVBUF, not set WRITE_BUFFER_WATER_MARK (same as 4.9.2)
* C. set SO_SNDBUF/SO_RCVBUF/WRITE_BUFFER_WATER_MARK
* D. remove SO_SNDBUF/SO_RCVBUF, set WRITE_BUFFER_WATER_MARK
The full rocketmq remoting ping test result
[Remoting.pdf](https://github.com/apache/rocketmq/files/8022269/Remoting.pdf)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] areyouok edited a comment on issue #3825: The TCP SO_SNDBUF/SO_RCVBUF is fixed and too small, decrease throughput greatly
Posted by GitBox <gi...@apache.org>.
areyouok edited a comment on issue #3825:
URL: https://github.com/apache/rocketmq/issues/3825#issuecomment-1032397075
Single producer/broker test, no consumer
* 1 producer , linux kernel 3.x
* 1 SyncMaster 1 Slave, linux kernel 4.x
* 18 queues for test topic
* producer has 400 threads
* network latency: 1.61ms
version | message body size | produce tps
-- | -- | --
Old | 250 | 88,000
Old | 4,000 | 18,200
Old | 16,000 | 4,860
Old | 100,000 | 750
New | 250 | 82,000
New | 4,000 | 70,000
New | 16,000 | 29,500
New | 100,000 | 5,950
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org