You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2022/03/24 05:55:52 UTC

[GitHub] [rocketmq] lwclover opened a new pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

lwclover opened a new pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981


   Async send mode, the default send executor add abort policy ThreadPoolExecutor.CallerRunsPolicy.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] codecov-commenter commented on pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981#issuecomment-1066446582


   # [Codecov](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#3981](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fdbf423) into [develop](https://codecov.io/gh/apache/rocketmq/commit/95202f1e6ffcd384b6aaefa5b4271b122d2eec21?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (95202f1) will **decrease** coverage by `0.15%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/rocketmq/pull/3981/graphs/tree.svg?width=650&height=150&src=pr&token=4w0sxP1wZv&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@              Coverage Diff              @@
   ##             develop    #3981      +/-   ##
   =============================================
   - Coverage      47.73%   47.57%   -0.16%     
   + Complexity      4977     4955      -22     
   =============================================
     Files            633      633              
     Lines          42579    42579              
     Branches        5589     5589              
   =============================================
   - Hits           20327    20259      -68     
   - Misses         19741    19807      +66     
   - Partials        2511     2513       +2     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...mq/client/impl/producer/DefaultMQProducerImpl.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jbGllbnQvaW1wbC9wcm9kdWNlci9EZWZhdWx0TVFQcm9kdWNlckltcGwuamF2YQ==) | `44.60% <ø> (-0.87%)` | :arrow_down: |
   | [...rg/apache/rocketmq/common/stats/StatsSnapshot.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvU3RhdHNTbmFwc2hvdC5qYXZh) | `84.61% <0.00%> (-15.39%)` | :arrow_down: |
   | [...rocketmq/broker/filtersrv/FilterServerManager.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvZmlsdGVyc3J2L0ZpbHRlclNlcnZlck1hbmFnZXIuamF2YQ==) | `20.00% <0.00%> (-14.29%)` | :arrow_down: |
   | [.../apache/rocketmq/common/stats/MomentStatsItem.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvTW9tZW50U3RhdHNJdGVtLmphdmE=) | `38.09% <0.00%> (-9.53%)` | :arrow_down: |
   | [...va/org/apache/rocketmq/store/FlushDiskWatcher.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL0ZsdXNoRGlza1dhdGNoZXIuamF2YQ==) | `81.25% <0.00%> (-9.38%)` | :arrow_down: |
   | [...ache/rocketmq/common/stats/MomentStatsItemSet.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvTW9tZW50U3RhdHNJdGVtU2V0LmphdmE=) | `39.13% <0.00%> (-8.70%)` | :arrow_down: |
   | [...in/java/org/apache/rocketmq/test/util/MQAdmin.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-dGVzdC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcm9ja2V0bXEvdGVzdC91dGlsL01RQWRtaW4uamF2YQ==) | `38.88% <0.00%> (-5.56%)` | :arrow_down: |
   | [...va/org/apache/rocketmq/common/stats/StatsItem.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvU3RhdHNJdGVtLmphdmE=) | `50.00% <0.00%> (-4.17%)` | :arrow_down: |
   | [...ketmq/common/protocol/body/ConsumerConnection.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vcHJvdG9jb2wvYm9keS9Db25zdW1lckNvbm5lY3Rpb24uamF2YQ==) | `95.83% <0.00%> (-4.17%)` | :arrow_down: |
   | [...org/apache/rocketmq/common/stats/StatsItemSet.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvU3RhdHNJdGVtU2V0LmphdmE=) | `43.28% <0.00%> (-2.99%)` | :arrow_down: |
   | ... and [13 more](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [95202f1...fdbf423](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] codecov-commenter edited a comment on pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981#issuecomment-1066446582


   # [Codecov](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#3981](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fdbf423) into [develop](https://codecov.io/gh/apache/rocketmq/commit/95202f1e6ffcd384b6aaefa5b4271b122d2eec21?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (95202f1) will **increase** coverage by `0.61%`.
   > The diff coverage is `n/a`.
   
   ```diff
   @@              Coverage Diff              @@
   ##             develop    #3981      +/-   ##
   =============================================
   + Coverage      47.73%   48.35%   +0.61%     
   - Complexity      4977     5075      +98     
   =============================================
     Files            633      633              
     Lines          42579    43322     +743     
     Branches        5589     5817     +228     
   =============================================
   + Hits           20327    20950     +623     
   - Misses         19741    19856     +115     
   - Partials        2511     2516       +5     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...mq/client/impl/producer/DefaultMQProducerImpl.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jbGllbnQvaW1wbC9wcm9kdWNlci9EZWZhdWx0TVFQcm9kdWNlckltcGwuamF2YQ==) | `47.98% <ø> (+2.50%)` | :arrow_up: |
   | [...rg/apache/rocketmq/common/stats/StatsSnapshot.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvU3RhdHNTbmFwc2hvdC5qYXZh) | `84.61% <0.00%> (-15.39%)` | :arrow_down: |
   | [.../apache/rocketmq/common/stats/MomentStatsItem.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvTW9tZW50U3RhdHNJdGVtLmphdmE=) | `38.09% <0.00%> (-9.53%)` | :arrow_down: |
   | [...va/org/apache/rocketmq/store/FlushDiskWatcher.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL0ZsdXNoRGlza1dhdGNoZXIuamF2YQ==) | `81.25% <0.00%> (-9.38%)` | :arrow_down: |
   | [...ache/rocketmq/common/stats/MomentStatsItemSet.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvTW9tZW50U3RhdHNJdGVtU2V0LmphdmE=) | `39.13% <0.00%> (-8.70%)` | :arrow_down: |
   | [...va/org/apache/rocketmq/common/stats/StatsItem.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvU3RhdHNJdGVtLmphdmE=) | `50.00% <0.00%> (-4.17%)` | :arrow_down: |
   | [...org/apache/rocketmq/common/stats/StatsItemSet.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvU3RhdHNJdGVtU2V0LmphdmE=) | `43.28% <0.00%> (-2.99%)` | :arrow_down: |
   | [...ava/org/apache/rocketmq/filter/util/BitsArray.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZmlsdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9maWx0ZXIvdXRpbC9CaXRzQXJyYXkuamF2YQ==) | `59.82% <0.00%> (-2.57%)` | :arrow_down: |
   | [.../rocketmq/broker/filter/ConsumerFilterManager.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvZmlsdGVyL0NvbnN1bWVyRmlsdGVyTWFuYWdlci5qYXZh) | `72.19% <0.00%> (-1.80%)` | :arrow_down: |
   | [...etmq/client/latency/LatencyFaultToleranceImpl.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jbGllbnQvbGF0ZW5jeS9MYXRlbmN5RmF1bHRUb2xlcmFuY2VJbXBsLmphdmE=) | `49.51% <0.00%> (-1.77%)` | :arrow_down: |
   | ... and [17 more](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [95202f1...fdbf423](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] lwclover closed pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
lwclover closed pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] odbozhou commented on a change in pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
odbozhou commented on a change in pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981#discussion_r834281945



##########
File path: client/src/main/java/org/apache/rocketmq/client/impl/producer/DefaultMQProducerImpl.java
##########
@@ -135,7 +135,8 @@ public DefaultMQProducerImpl(final DefaultMQProducer defaultMQProducer, RPCHook
                 public Thread newThread(Runnable r) {
                     return new Thread(r, "AsyncSenderExecutor_" + this.threadIndex.incrementAndGet());
                 }
-            });
+            },
+            new ThreadPoolExecutor.CallerRunsPolicy());

Review comment:
       我觉得你说的这种场景，可能增加一个producer从而增加线程数，提升发送处理速度，可能更有利于解决这个问题。




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] coveralls edited a comment on pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981#issuecomment-1066446442


   
   [![Coverage Status](https://coveralls.io/builds/47643558/badge)](https://coveralls.io/builds/47643558)
   
   Coverage increased (+0.02%) to 51.907% when pulling **fdbf4238c89a3667bbb04c56b6a8c338084d6af9 on lwclover:develop** into **c1aeca291ea686a2b7f01ffec30132f5a370a2ab on apache:develop**.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] odbozhou commented on a change in pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
odbozhou commented on a change in pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981#discussion_r832883297



##########
File path: client/src/main/java/org/apache/rocketmq/client/impl/producer/DefaultMQProducerImpl.java
##########
@@ -135,7 +135,8 @@ public DefaultMQProducerImpl(final DefaultMQProducer defaultMQProducer, RPCHook
                 public Thread newThread(Runnable r) {
                     return new Thread(r, "AsyncSenderExecutor_" + this.threadIndex.incrementAndGet());
                 }
-            });
+            },
+            new ThreadPoolExecutor.CallerRunsPolicy());

Review comment:
       why user CallerRunsPolicy replace default AbortPolicy？




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] lwclover commented on a change in pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
lwclover commented on a change in pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981#discussion_r833958324



##########
File path: client/src/main/java/org/apache/rocketmq/client/impl/producer/DefaultMQProducerImpl.java
##########
@@ -135,7 +135,8 @@ public DefaultMQProducerImpl(final DefaultMQProducer defaultMQProducer, RPCHook
                 public Thread newThread(Runnable r) {
                     return new Thread(r, "AsyncSenderExecutor_" + this.threadIndex.incrementAndGet());
                 }
-            });
+            },
+            new ThreadPoolExecutor.CallerRunsPolicy());

Review comment:
       rocketmq默认的异步线程池是大小和cpu核数相等，阻塞队列默认是5w，我们生产应用使用k8s部署一般cpu也就2个核左右，当时是准备异步发送8w+的消息。
   出现了RejectedExecutionException异常，这个倒是容易理解，发送到线程池的速度肯定是快于producer发送到broker的速度的，当时我建议业务set一个自定义线程池并配置淘汰策略为CallerRunsPolicy，解决了这个问题。
   
   如果发生了Full gc和发送慢的问题，我觉得应该继续优化jvm参数和扩展broker性能，或者减少阻塞队列的容量解决。




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] odbozhou commented on a change in pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
odbozhou commented on a change in pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981#discussion_r833855946



##########
File path: client/src/main/java/org/apache/rocketmq/client/impl/producer/DefaultMQProducerImpl.java
##########
@@ -135,7 +135,8 @@ public DefaultMQProducerImpl(final DefaultMQProducer defaultMQProducer, RPCHook
                 public Thread newThread(Runnable r) {
                     return new Thread(r, "AsyncSenderExecutor_" + this.threadIndex.incrementAndGet());
                 }
-            });
+            },
+            new ThreadPoolExecutor.CallerRunsPolicy());

Review comment:
       In my opinion, if it is to limit the flow and reject redundant tasks, the default AbortPolicy should be enough, and CallerRunsPolicy will run the task, in which case the task is discarded.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] coveralls commented on pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
coveralls commented on pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981#issuecomment-1066446436






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] odbozhou commented on a change in pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
odbozhou commented on a change in pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981#discussion_r833950779



##########
File path: client/src/main/java/org/apache/rocketmq/client/impl/producer/DefaultMQProducerImpl.java
##########
@@ -135,7 +135,8 @@ public DefaultMQProducerImpl(final DefaultMQProducer defaultMQProducer, RPCHook
                 public Thread newThread(Runnable r) {
                     return new Thread(r, "AsyncSenderExecutor_" + this.threadIndex.incrementAndGet());
                 }
-            });
+            },
+            new ThreadPoolExecutor.CallerRunsPolicy());

Review comment:
       线程池blockingqueue已经提供了很大的任务挤压数量，如果触发了线程池的拒绝策略，说明异步发送任务量已经非常大，客户端压力非常大了，快速失败也是对客户端的保护，不及时拒绝新的任务，任务一直挤压，可能导致更严重的问题（fullgc，发送慢等等）。




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] lwclover commented on a change in pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
lwclover commented on a change in pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981#discussion_r833844473



##########
File path: client/src/main/java/org/apache/rocketmq/client/impl/producer/DefaultMQProducerImpl.java
##########
@@ -135,7 +135,8 @@ public DefaultMQProducerImpl(final DefaultMQProducer defaultMQProducer, RPCHook
                 public Thread newThread(Runnable r) {
                     return new Thread(r, "AsyncSenderExecutor_" + this.threadIndex.incrementAndGet());
                 }
-            });
+            },
+            new ThreadPoolExecutor.CallerRunsPolicy());

Review comment:
       The CallerRunsPolicy is used to limit traffic.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] lwclover commented on a change in pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
lwclover commented on a change in pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981#discussion_r833926435



##########
File path: client/src/main/java/org/apache/rocketmq/client/impl/producer/DefaultMQProducerImpl.java
##########
@@ -135,7 +135,8 @@ public DefaultMQProducerImpl(final DefaultMQProducer defaultMQProducer, RPCHook
                 public Thread newThread(Runnable r) {
                     return new Thread(r, "AsyncSenderExecutor_" + this.threadIndex.incrementAndGet());
                 }
-            });
+            },
+            new ThreadPoolExecutor.CallerRunsPolicy());

Review comment:
       本质上还是想发送成功的，默认的线程池AbortPolicy当阻塞队列满时抛出异常；而CallerRunsPolicy提供简单的反馈控制机制，能够减缓新任务的提交速度。我的出发点是尽量发送成功。




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] lwclover commented on a change in pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
lwclover commented on a change in pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981#discussion_r833842407



##########
File path: client/src/main/java/org/apache/rocketmq/client/impl/producer/DefaultMQProducerImpl.java
##########
@@ -135,7 +135,8 @@ public DefaultMQProducerImpl(final DefaultMQProducer defaultMQProducer, RPCHook
                 public Thread newThread(Runnable r) {
                     return new Thread(r, "AsyncSenderExecutor_" + this.threadIndex.incrementAndGet());
                 }
-            });
+            },
+            new ThreadPoolExecutor.CallerRunsPolicy());

Review comment:
       Due to the large number of messages in the production asynchronous sending mode, the block queue capacity was full and subsequent sending failed
   
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] lwclover commented on a change in pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
lwclover commented on a change in pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981#discussion_r833958324



##########
File path: client/src/main/java/org/apache/rocketmq/client/impl/producer/DefaultMQProducerImpl.java
##########
@@ -135,7 +135,8 @@ public DefaultMQProducerImpl(final DefaultMQProducer defaultMQProducer, RPCHook
                 public Thread newThread(Runnable r) {
                     return new Thread(r, "AsyncSenderExecutor_" + this.threadIndex.incrementAndGet());
                 }
-            });
+            },
+            new ThreadPoolExecutor.CallerRunsPolicy());

Review comment:
       rocketmq默认的异步线程池是大小和cpu核数相等，阻塞队列默认是5w，我们生产应用使用k8s部署一般cpu也就2个核左右，当时是准备异步发送8w+的消息。
   出现了RejectedExecutionException异常，这个倒是容易理解，发送到线程池的速度肯定是快于producer发送到broker的速度的，当时我建议业务set一个自定义线程池并配置淘汰策略为CallerRunsPolicy，解决了这个问题。
   
   如果发生了Full gc和发送慢的问题，我觉得应该继续优化jvm和broker解决。




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] lwclover commented on a change in pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
lwclover commented on a change in pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981#discussion_r833958324



##########
File path: client/src/main/java/org/apache/rocketmq/client/impl/producer/DefaultMQProducerImpl.java
##########
@@ -135,7 +135,8 @@ public DefaultMQProducerImpl(final DefaultMQProducer defaultMQProducer, RPCHook
                 public Thread newThread(Runnable r) {
                     return new Thread(r, "AsyncSenderExecutor_" + this.threadIndex.incrementAndGet());
                 }
-            });
+            },
+            new ThreadPoolExecutor.CallerRunsPolicy());

Review comment:
       rocketmq默认的异步线程池是大小和cpu核数相等，阻塞队列默认是5w，我们生产应用使用k8s部署一般cpu也就2个核左右，当时是准备异步发送8w+的消息。
   出现了RejectedExecutionException异常，这个倒是容易理解，发送到线程池的速度肯定是快于producer发送到broker的速度的，当时我建议业务set一个自定义线程池并配置淘汰策略为CallerRunsPolicy，解决这个问题。
   
   如果发生了Full gc和发送慢的问题，我觉得应该继续优化jvm和broker解决。




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] lwclover commented on a change in pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
lwclover commented on a change in pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981#discussion_r834869995



##########
File path: client/src/main/java/org/apache/rocketmq/client/impl/producer/DefaultMQProducerImpl.java
##########
@@ -135,7 +135,8 @@ public DefaultMQProducerImpl(final DefaultMQProducer defaultMQProducer, RPCHook
                 public Thread newThread(Runnable r) {
                     return new Thread(r, "AsyncSenderExecutor_" + this.threadIndex.incrementAndGet());
                 }
-            });
+            },
+            new ThreadPoolExecutor.CallerRunsPolicy());

Review comment:
       我个人认为设置CallerRunsPolicy可减少一个风险点。咱们等等看看其他人有没有更好的方案，这个pr在放放




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] codecov-commenter edited a comment on pull request #3981: [ISSUE #3982]Async send mode, the default send executor add abort policy

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3981:
URL: https://github.com/apache/rocketmq/pull/3981#issuecomment-1066446582


   # [Codecov](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#3981](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fdbf423) into [develop](https://codecov.io/gh/apache/rocketmq/commit/95202f1e6ffcd384b6aaefa5b4271b122d2eec21?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (95202f1) will **increase** coverage by `0.60%`.
   > The diff coverage is `n/a`.
   
   ```diff
   @@              Coverage Diff              @@
   ##             develop    #3981      +/-   ##
   =============================================
   + Coverage      47.73%   48.34%   +0.60%     
   - Complexity      4977     5074      +97     
   =============================================
     Files            633      633              
     Lines          42579    43322     +743     
     Branches        5589     5817     +228     
   =============================================
   + Hits           20327    20942     +615     
   - Misses         19741    19863     +122     
   - Partials        2511     2517       +6     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...mq/client/impl/producer/DefaultMQProducerImpl.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jbGllbnQvaW1wbC9wcm9kdWNlci9EZWZhdWx0TVFQcm9kdWNlckltcGwuamF2YQ==) | `47.98% <ø> (+2.50%)` | :arrow_up: |
   | [...rg/apache/rocketmq/common/stats/StatsSnapshot.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvU3RhdHNTbmFwc2hvdC5qYXZh) | `84.61% <0.00%> (-15.39%)` | :arrow_down: |
   | [.../apache/rocketmq/common/stats/MomentStatsItem.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvTW9tZW50U3RhdHNJdGVtLmphdmE=) | `38.09% <0.00%> (-9.53%)` | :arrow_down: |
   | [...va/org/apache/rocketmq/store/FlushDiskWatcher.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL0ZsdXNoRGlza1dhdGNoZXIuamF2YQ==) | `81.25% <0.00%> (-9.38%)` | :arrow_down: |
   | [...ache/rocketmq/common/stats/MomentStatsItemSet.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvTW9tZW50U3RhdHNJdGVtU2V0LmphdmE=) | `39.13% <0.00%> (-8.70%)` | :arrow_down: |
   | [...in/java/org/apache/rocketmq/test/util/MQAdmin.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-dGVzdC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcm9ja2V0bXEvdGVzdC91dGlsL01RQWRtaW4uamF2YQ==) | `38.88% <0.00%> (-5.56%)` | :arrow_down: |
   | [...va/org/apache/rocketmq/common/stats/StatsItem.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvU3RhdHNJdGVtLmphdmE=) | `50.00% <0.00%> (-4.17%)` | :arrow_down: |
   | [...ketmq/common/protocol/body/ConsumerConnection.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vcHJvdG9jb2wvYm9keS9Db25zdW1lckNvbm5lY3Rpb24uamF2YQ==) | `95.83% <0.00%> (-4.17%)` | :arrow_down: |
   | [...org/apache/rocketmq/common/stats/StatsItemSet.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvU3RhdHNJdGVtU2V0LmphdmE=) | `43.28% <0.00%> (-2.99%)` | :arrow_down: |
   | [...ava/org/apache/rocketmq/filter/util/BitsArray.java](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZmlsdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9maWx0ZXIvdXRpbC9CaXRzQXJyYXkuamF2YQ==) | `59.82% <0.00%> (-2.57%)` | :arrow_down: |
   | ... and [18 more](https://codecov.io/gh/apache/rocketmq/pull/3981/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [95202f1...fdbf423](https://codecov.io/gh/apache/rocketmq/pull/3981?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org