You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by "Jun Rao (JIRA)" <ji...@apache.org> on 2018/10/26 22:55:00 UTC

[jira] [Commented] (KAFKA-7557) optimize LogManager.truncateFullyAndStartAt()

    [ https://issues.apache.org/jira/browse/KAFKA-7557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16665737#comment-16665737 ] 

Jun Rao commented on KAFKA-7557:
--------------------------------

To improve this, instead of calling Log.deleteSnapshotsAfterRecoveryPointCheckpoint() on all logs, we could probably call only the one that's being truncated.

> optimize LogManager.truncateFullyAndStartAt()
> ---------------------------------------------
>
>                 Key: KAFKA-7557
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7557
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 2.0.0, 2.1.0
>            Reporter: Jun Rao
>            Priority: Major
>
> When a ReplicaFetcherThread calls LogManager.truncateFullyAndStartAt() for a partition, we call LogManager.checkpointLogRecoveryOffsetsInDir() and then Log.deleteSnapshotsAfterRecoveryPointCheckpoint() on all the logs in that directory. This requires listing all the files in each log dir to figure out the snapshot files. If some logs have many log segment files. This could take some time. The can potentially block a replica fetcher thread, which indirectly causes the request handler threads to be blocked.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)