You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Emi (Jira)" <ji...@apache.org> on 2021/06/14 07:37:00 UTC
[jira] [Updated] (KAFKA-12946) __consumer_offsets topic with very
big partitions
[ https://issues.apache.org/jira/browse/KAFKA-12946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Emi updated KAFKA-12946:
------------------------
Description:
I am using Kafka 2.0.0 with java 8u191
There is a partitions of the __consumer_offsets topic that is 600 GB with 6000 segments older than 4 months. Other partitions of that topic are small: 20-30MB.
There are 60 consumer groups, 90 topics and 100 partitions per topic.
There aren't errors in the logs. From the log of the logcleaner, I can see that partition is never touched from the logcleaner thread for the compaction, but it only add new segments.
How is this possible?
There was another partition with the same problem, but after some months it has been compacted. Now there is only one partition with this problem, but this is bigger and keep growing
My settings:
{{offsets.commit.required.acks = -1}}
{{[offsets.commit.timeout.ms|http://offsets.commit.timeout.ms/]}} = 5000
{{offsets.load.buffer.size = 5242880}}
{{[offsets.retention.check.interval.ms|http://offsets.retention.check.interval.ms/]}} = 600000
{{offsets.retention.minutes = 10080}}
{{offsets.topic.compression.codec = 0}}
{{offsets.topic.num.partitions = 50}}
{{offsets.topic.replication.factor = 3}}
{{offsets.topic.segment.bytes = 104857600}}
was:
I am using Kafka 2.0.0 with java 8u191
There is a partitions of the __consumer_offsets topic that is 600 GB with 6000 segments older than 4 months. Other partitions of that topic are small: 20-30MB.
There are 60 consumer groups, 90 topics and 100 partitions per topic.
There aren't errors in the logs. From the log of the logcleaner, I can see that partition is never touched from the logcleaner thread for the compaction, but it only add new segments.
How is this possible?
There was another partition with the same problem, but after some months it has been compacted. Now there is only one partition with this problem, but this is bigger and keep growing
My settings:
{{offsets.commit.required.acks = -1}}
{{[offsets.commit.timeout.ms|http://offsets.commit.timeout.ms/]}}{{ = 5000}}
{{offsets.load.buffer.size = 5242880}}
{{[offsets.retention.check.interval.ms|http://offsets.retention.check.interval.ms/]}}{{ = 600000}}
{{offsets.retention.minutes = 10080}}
{{offsets.topic.compression.codec = 0}}
{{offsets.topic.num.partitions = 50}}
{{offsets.topic.replication.factor = 3}}
{{offsets.topic.segment.bytes = 104857600}}
> __consumer_offsets topic with very big partitions
> -------------------------------------------------
>
> Key: KAFKA-12946
> URL: https://issues.apache.org/jira/browse/KAFKA-12946
> Project: Kafka
> Issue Type: Bug
> Components: log cleaner
> Affects Versions: 2.0.0
> Reporter: Emi
> Priority: Critical
>
> I am using Kafka 2.0.0 with java 8u191
> There is a partitions of the __consumer_offsets topic that is 600 GB with 6000 segments older than 4 months. Other partitions of that topic are small: 20-30MB.
> There are 60 consumer groups, 90 topics and 100 partitions per topic.
> There aren't errors in the logs. From the log of the logcleaner, I can see that partition is never touched from the logcleaner thread for the compaction, but it only add new segments.
> How is this possible?
> There was another partition with the same problem, but after some months it has been compacted. Now there is only one partition with this problem, but this is bigger and keep growing
> My settings:
> {{offsets.commit.required.acks = -1}}
> {{[offsets.commit.timeout.ms|http://offsets.commit.timeout.ms/]}} = 5000
> {{offsets.load.buffer.size = 5242880}}
> {{[offsets.retention.check.interval.ms|http://offsets.retention.check.interval.ms/]}} = 600000
> {{offsets.retention.minutes = 10080}}
> {{offsets.topic.compression.codec = 0}}
> {{offsets.topic.num.partitions = 50}}
> {{offsets.topic.replication.factor = 3}}
> {{offsets.topic.segment.bytes = 104857600}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)