You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Roland Sommer (Jira)" <ji...@apache.org> on 2024/05/16 12:48:00 UTC

[jira] [Created] (KAFKA-16782) Some partition's segments are suddenly not deleted anymore

Roland Sommer created KAFKA-16782:
-------------------------------------

             Summary: Some partition's segments are suddenly not deleted anymore
                 Key: KAFKA-16782
                 URL: https://issues.apache.org/jira/browse/KAFKA-16782
             Project: Kafka
          Issue Type: Bug
            Reporter: Roland Sommer


I recently discovered an odd behaviour in one of our kafka clusters (KRaft-based, v3.7.0):

We have a topic for distributed log collection with 48 partitions. Retention is set to 84 hours, we have the default {{cleanup.policy=delete}} in place. For all but two partitions this works as expected. In two partition directories there are files going back to january and consuming the specific partitions yields data from january (showing it's not only the files lying around, they are actually processed).

Topic settings as per {{kafka-topics.sh --describe}}:

{{Topic: syslog TopicId: AeJLnYPnQFOtMc0ZjpH7sw PartitionCount: 48 ReplicationFactor: 2 Configs: compression.type=snappy,cleanup.policy=delete,segment.bytes=1073741824,retention.ms=302400000,max.message.bytes=2097152}}

Searching the cluster logs, there is no indicator of what could be the reason here (at least I did not spot anything suspicious up until now). Up to the time were deletion stopped, there are log entries showing the deleteion of old log segments, but that simply stopped. As far as I can see, there has not been any change on the cluster at that point.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)