[jira] [Commented] (ROCKETMQ-294) Do flow control on the number and size dimensions when pull message

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ROCKETMQ-294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16178940#comment-16178940 ] 

ASF GitHub Bot commented on ROCKETMQ-294:
-----------------------------------------

GitHub user zhouxinyu opened a pull request:

    https://github.com/apache/incubator-rocketmq/pull/171

    [ROCKETMQ-294] Do flow control on the number and size dimensions when pull message

    ## What is the purpose of the change
    Current flow control strategy only support on Queue level, each message queue can cache 1000 message by default.
    
    When lots of message queue are assigned to one consumer instance, the consumer will cache too many messages, may cause OOM exception.
    
    On the other hand, only control the message amount is not enough, should support do flow control on the number and size dimensions, on Topic and Queue level.
    
    ## Brief changelog
    
    Do flow control on the number and size dimensions when pulling message
    
    ## Verifying this change
    
    This PR added three parameters, plus the old one, there are 4 parameters for flow control:
    
    * pullThresholdForQueue, flow control threshold on queue level, each message queue will cache at most 1000 messages by default.
    * pullThresholdForTopic, flow control threshold on topic level, default value is -1(Unlimited)
    * pullThresholdSizeForQueue, limit the cached message size on queue level, each message queue will cache at most 100 MiB messages by default.
    * pullThresholdSizeForTopic, limit the cached message size on topic level, the default value is -1 MiB(Unlimited).
    
    See the javadoc for details, you can change the value, and observe the log to ensure it works.
    
    Follow this checklist to help us incorporate your contribution quickly and easily:
    
    - [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/browse/ROCKETMQ-294) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue. 
    - [x] Format the pull request title like `[ROCKETMQ-XXX] Fix UnknownException when host config not exist`. Each commit in the pull request should have a meaningful subject line and body.
    - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
    - [x] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in [test module](https://github.com/apache/incubator-rocketmq/tree/master/test).
    - [x] Run `mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle` to make sure basic checks pass. Run `mvn clean install -DskipITs` to make sure unit-test pass. Run `mvn clean test-compile failsafe:integration-test`  to make sure integration-test pass.
    - [x] If this contribution is large, please file an [Apache Individual Contributor License Agreement](http://www.apache.org/licenses/#clas).


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zhouxinyu/incubator-rocketmq ROCKETMQ-294

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-rocketmq/pull/171.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #171
    
----
commit 8878a21e2d0a87ac0d1ae59483eabcc34fdafb07
Author: yukon <yu...@apache.org>
Date:   2017-09-25T07:55:55Z

    [ROCKETMQ-294] Do flow control on the number and size dimensions when pull message

----


> Do flow control on the number and size dimensions when pull message
> -------------------------------------------------------------------
>
>                 Key: ROCKETMQ-294
>                 URL: https://issues.apache.org/jira/browse/ROCKETMQ-294
>             Project: Apache RocketMQ
>          Issue Type: Improvement
>          Components: rocketmq-client
>            Reporter: yukon
>            Assignee: yukon
>             Fix For: 4.2.0-incubating
>
>
> *Motivations*
> Current flow control strategy only support on Queue level, each message queue can cache 1000 message by default. 
> When lots of message queue are assigned to one consumer instance, the consumer will cache too many messages, may cause OOM exception.
> On the other hand, only control the message amount is not enough, should support do flow control on the number and size dimensions, on Topic and Queue level.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)