[jira] [Commented] (ZOOKEEPER-3356) Request throttling in Netty is not working as expected and could cause direct buffer OOM issue

["Hadoop QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ZOOKEEPER-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16890432#comment-16890432 ] 

Hadoop QA commented on ZOOKEEPER-3356:
--------------------------------------

+1 overall.  GitHub Pull Request  Build
      

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 12 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 3.0.1) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    +1 core tests.  The patch passed core unit tests.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/4066//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/4066//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/4066//console

This message is automatically generated.

> Request throttling in Netty is not working as expected and could cause direct buffer OOM issue 
> -----------------------------------------------------------------------------------------------
>
>                 Key: ZOOKEEPER-3356
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3356
>             Project: ZooKeeper
>          Issue Type: Bug
>          Components: server
>    Affects Versions: 3.5.4, 3.6.0
>            Reporter: Fangmin Lv
>            Assignee: Fangmin Lv
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.6.0
>
>          Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> The current implementation of Netty enable/disable recv logic may cause the direct buffer OOM because we may enable read a large chunk of packets and disabled again after consuming a single ZK request. We have seen this problem on prod occasionally.
>  
> Need a more advanced flow control in Netty instead of using AUTO_READ. Have improved it internally by enable/disable recv based on the queuedBuffer size, will upstream this soon.
>  
> With this implementation, the max Netty queued buffer size (direct memory usage) will be 2 * recv_buffer size. It's not the per message size because in epoll ET mode it will try to read until the socket is empty, and because of SslHandler will trigger another read when it's not a full encrypt packet and haven't issued any decrypt message.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)