[jira] [Commented] (KAFKA-5250) handleFetchRequest should do down conversion after throttling

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-5250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16012367#comment-16012367 ] 

ASF GitHub Bot commented on KAFKA-5250:
---------------------------------------

GitHub user rajinisivaram opened a pull request:

    https://github.com/apache/kafka/pull/3068

    KAFKA-5250: Do fetch down conversion after throttling

    Perform down conversion after throttling to avoid retaining messages in memory during throttling since this could result in OOM. Also update bytesOut metrics after throttling when the response is about to be sent.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/rajinisivaram/kafka KAFKA-5250

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/3068.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3068
    
----
commit a5d0df74214480929f800773dc2aee8d0a4b0bc6
Author: Rajini Sivaram <ra...@googlemail.com>
Date:   2017-05-16T12:49:21Z

    KAFKA-5250: Do fetch down conversion after throttling

----


> handleFetchRequest should do down conversion after throttling
> -------------------------------------------------------------
>
>                 Key: KAFKA-5250
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5250
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Ismael Juma
>            Assignee: Rajini Sivaram
>             Fix For: 0.11.0.0
>
>
> We currently do down conversion before throttling. This is good from the perspective of getting the correct message size, but it means that we can cause OOMs due to excessive memory retention. That is, by performing down conversion, we are loading the records into the heap even though we are not ready to send them yet.
> It would be preferable to throttle before down conversion.
> In addition, we currently updates bytesOut before throttling. We should do it after throttling as well.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)