You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Badai Aqrandista (Jira)" <ji...@apache.org> on 2021/07/07 23:02:00 UTC
[jira] [Assigned] (KAFKA-7760) Add broker configuration to set
minimum value for segment.bytes and segment.ms
[ https://issues.apache.org/jira/browse/KAFKA-7760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Badai Aqrandista reassigned KAFKA-7760:
---------------------------------------
Assignee: Badai Aqrandista (was: Dulvin Witharane)
> Add broker configuration to set minimum value for segment.bytes and segment.ms
> ------------------------------------------------------------------------------
>
> Key: KAFKA-7760
> URL: https://issues.apache.org/jira/browse/KAFKA-7760
> Project: Kafka
> Issue Type: Improvement
> Reporter: Badai Aqrandista
> Assignee: Badai Aqrandista
> Priority: Major
> Labels: kip, newbie
>
> If someone set segment.bytes or segment.ms at topic level to a very small value (e.g. segment.bytes=1000 or segment.ms=1000), Kafka will generate a very high number of segment files. This can bring down the whole broker due to hitting the maximum open file (for log) or maximum number of mmap-ed file (for index).
> To prevent that from happening, I would like to suggest adding two new items to the broker configuration:
> * min.topic.segment.bytes, defaults to 1048576: The minimum value for segment.bytes. When someone sets topic configuration segment.bytes to a value lower than this, Kafka throws an error INVALID VALUE.
> * min.topic.segment.ms, defaults to 3600000: The minimum value for segment.ms. When someone sets topic configuration segment.ms to a value lower than this, Kafka throws an error INVALID VALUE.
> Thanks
> Badai
--
This message was sent by Atlassian Jira
(v8.3.4#803005)