You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "ChenLin (Jira)" <ji...@apache.org> on 2019/08/29 03:25:00 UTC
[jira] [Comment Edited] (KAFKA-8832) We should limit the maximum
size read by a fetch request on the kafka server.
[ https://issues.apache.org/jira/browse/KAFKA-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918238#comment-16918238 ]
ChenLin edited comment on KAFKA-8832 at 8/29/19 3:24 AM:
---------------------------------------------------------
[~guozhang] I did a test and then dumped the jvm memory at the time. Found that there is a lot of buffer data in memory：
!image-2019-08-29-11-23-53-155.png!
was (Author: lordchen):
[~guozhang] I did a test and then dumped the jvm memory at the time. Found that there is a lot of buffer data in memory： !image-2019-08-29-11-03-37-693.png!
> We should limit the maximum size read by a fetch request on the kafka server.
> -----------------------------------------------------------------------------
>
> Key: KAFKA-8832
> URL: https://issues.apache.org/jira/browse/KAFKA-8832
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 2.3.0, 2.2.1
> Reporter: ChenLin
> Priority: Major
> Labels: needs-kip
> Attachments: image-2019-08-25-15-31-56-707.png, image-2019-08-25-15-42-24-379.png, image-2019-08-29-11-01-04-147.png, image-2019-08-29-11-01-17-347.png, image-2019-08-29-11-02-01-477.png, image-2019-08-29-11-03-37-693.png, image-2019-08-29-11-21-49-998.png, image-2019-08-29-11-23-53-155.png
>
>
> I found that kafka is not on the server side, limiting the amount of data read per fetch request. This may cause the kafka server program to report an error: OutOfMemory. Due to unreasonable client configuration, fetch.message.max.bytes configuration is too large, such as 100M, because the kafka server receives a lot of fetch requests at a certain moment, causing the server to report an error: OutOfMemory。So I think this is a bug。
> !image-2019-08-29-11-02-01-477.png!
> !image-2019-08-25-15-42-24-379.png!
> !image-2019-08-25-15-31-56-707.png!
--
This message was sent by Atlassian Jira
(v8.3.2#803003)