You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Parag Shah <ps...@proofpoint.com> on 2014/11/20 19:00:04 UTC

TTL changes - Are they retroactive?

Hi all,

    I had a question about TTL changes to a topic’s configuration. When I make the change and restart kafka, will kafka apply the changes to all existing messages on disk retroactively or would it do it only for new messages entering the topic?

Regards
Parag

Re: TTL changes - Are they retroactive?

Posted by Daniel Compton <de...@danielcompton.net>.

Hi Parag

Just to expand on Jun’s comment, log retention and deletion is at the segment level, not the message level. Because it’s at the segment level I would avoid using the term TTL, as that would normally be applied to individual items.

Every log.retention.check.interval.ms (default 5 minutes), the Kafka broker will check whether any log segment is eligible for deletion. The whole log segment must be eligible for deletion based on the retention policy so some messages in the segment will be older than the log.retention.hours (default 7 days) set in the Kafka config.

This isn’t a big deal normally, but is a bit different from other messaging systems so I thought it was worth pointing out the difference.

Daniel.

> On 21/11/2014, at 12:39 pm, Jun Rao <ju...@gmail.com> wrote:
> 
> The retention configure change will be applied on existing data too.
> 
> Thanks,
> 
> Jun
> 
> On Thu, Nov 20, 2014 at 10:00 AM, Parag Shah <ps...@proofpoint.com> wrote:
> 
>> Hi all,
>> 
>>    I had a question about TTL changes to a topic’s configuration. When I
>> make the change and restart kafka, will kafka apply the changes to all
>> existing messages on disk retroactively or would it do it only for new
>> messages entering the topic?
>> 
>> Regards
>> Parag
>>

Re: TTL changes - Are they retroactive?

Posted by Jun Rao <ju...@gmail.com>.

The retention configure change will be applied on existing data too.

Thanks,

Jun

On Thu, Nov 20, 2014 at 10:00 AM, Parag Shah <ps...@proofpoint.com> wrote:

> Hi all,
>
>     I had a question about TTL changes to a topic’s configuration. When I
> make the change and restart kafka, will kafka apply the changes to all
> existing messages on disk retroactively or would it do it only for new
> messages entering the topic?
>
> Regards
> Parag
>