You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Jiangtao Liu (Jira)" <ji...@apache.org> on 2020/04/07 21:40:00 UTC
[jira] [Updated] (KAFKA-8270) Kafka retention hour is not working
[ https://issues.apache.org/jira/browse/KAFKA-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jiangtao Liu updated KAFKA-8270:
--------------------------------
Description:
Kafka l cannot be deleted after the configured retention hours (12 hours for log retention).
What's our Kafka cluster look like?
There are 6 brokers deployed with Kafka version 1.1.1.
Is it reproducible?
I am not sure since our Kafka cluster is working well over 1.5 years without retention issue until 4/13/2019 ~ 4/20/2019.
is it related to the active segment?
as I know Kafka will not delete an active segment, my case those old logs are not activated, they should be inactivated.
What's the current status?
Those old logs have been deleted after I manually ran rolling restart Kafka servers with retention hours adjustment (Ideally I tried this solution aimed to{color:#333333} force retention hours work, not really want to adjust the retention hours, finally the solution it's working, but not immediately, I remember the retention start work after couples of hours after applying the change and rolling restart Kafka servers{color}.), now our Kafka storage is back to normal, please check the screenshot attached with this ticket.
A sample old log added here for better understanding of the retention hours not working issue.
```
// it has been there from 4/12
-rw-r--r-- 1 root root 136866627{color:#d04437} Apr 12{color} 04:33 00000000002581377820.log
// It was still being opened by Kafka when I check it with the tool lsof on {color:#d04437}4/19/2019 before server rolling restart with retention hours adjustment{color}.
java 20281 0 1678u REG 202,32 136866627 1074562295 /kafka/data/.../00000000002581377820.log
```
was:
Kafka log cannot be deleted after the configured retention hours (12 hours for log retention).
What's our Kafka cluster look like?
There are 6 brokers deployed with Kafka version 1.1.1.
Is it reproducible?
I am not sure since our Kafka cluster is working well over 1.5 years without retention issue until 4/13/2019 ~ 4/20/2019.
is it related to the active segment?
as I know Kafka will not delete an active segment, my case those old logs are not activated, they should be inactivated.
What's the current status?
Those old logs have been deleted after I manually ran rolling restart Kafka servers with retention hours adjustment (Ideally I tried this solution aimed to{color:#333333} force retention hours work, not really want to adjust the retention hours, finally the solution it's working, but not immediately, I remember the retention start work after couples of hours after applying the change and rolling restart Kafka servers{color}.), now our Kafka storage is back to normal, please check the screenshot attached with this ticket.
A sample old log added here for better understanding of the retention hours not working issue.
```
// it has been there from 4/12
-rw-r--r-- 1 root root 136866627{color:#d04437} Apr 12{color} 04:33 00000000002581377820.log
// It was still being opened by Kafka when I check it with the tool lsof on {color:#d04437}4/19/2019 before server rolling restart with retention hours adjustment{color}.
java 20281 0 1678u REG 202,32 136866627 1074562295 /kafka/data/.../00000000002581377820.log
```
> Kafka retention hour is not working
> -----------------------------------
>
> Key: KAFKA-8270
> URL: https://issues.apache.org/jira/browse/KAFKA-8270
> Project: Kafka
> Issue Type: Bug
> Components: log
> Reporter: Jiangtao Liu
> Priority: Major
> Labels: storage
> Attachments: Screen Shot 2019-04-20 at 10.57.59 PM.png
>
>
> Kafka l cannot be deleted after the configured retention hours (12 hours for log retention).
> What's our Kafka cluster look like?
> There are 6 brokers deployed with Kafka version 1.1.1.
>
> Is it reproducible?
> I am not sure since our Kafka cluster is working well over 1.5 years without retention issue until 4/13/2019 ~ 4/20/2019.
>
> is it related to the active segment?
> as I know Kafka will not delete an active segment, my case those old logs are not activated, they should be inactivated.
>
> What's the current status?
> Those old logs have been deleted after I manually ran rolling restart Kafka servers with retention hours adjustment (Ideally I tried this solution aimed to{color:#333333} force retention hours work, not really want to adjust the retention hours, finally the solution it's working, but not immediately, I remember the retention start work after couples of hours after applying the change and rolling restart Kafka servers{color}.), now our Kafka storage is back to normal, please check the screenshot attached with this ticket.
> A sample old log added here for better understanding of the retention hours not working issue.
> ```
> // it has been there from 4/12
> -rw-r--r-- 1 root root 136866627{color:#d04437} Apr 12{color} 04:33 00000000002581377820.log
> // It was still being opened by Kafka when I check it with the tool lsof on {color:#d04437}4/19/2019 before server rolling restart with retention hours adjustment{color}.
> java 20281 0 1678u REG 202,32 136866627 1074562295 /kafka/data/.../00000000002581377820.log
> ```
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)