Re: retention.ms is not respected after upgrade to 0.8.2

[Jaikiran Pai <ja...@gmail.com>]

I can't see anything obvious wrong in those configs or the code (after 
just a brief look). Are you sure the system on which you are running 
Kafka has its date/time correctly set?

-Jaikiran
On Monday 29 June 2015 12:06 PM, Krzysztof Zarzycki wrote:
> Greetings!
> I have problem with Kafka. I had a cluster of 3 brokers in version 0.8.1. I
> have a very important topic with raw events, that had a config
> retention.ms={365
> days in ms} .
> It all worked, fine, data was not being deleted.
> But now I upgraded all brokers to 0.8.2 and suddenly  brokers delete the
> data! They don't respect retention.ms.
> I have no other settings around retention set: global log.retention.hours
> is set to default 168, log.retention.bytes is not set.
>
> Some more info:
> 1. I tried looking into ZK config and it looks fine:
> $ get /kafka081/config/topics/my_topic
> {"version":1,"config":{"retention.ms":"31536000000"}}
> cZxid = 0xa0006412c
> ctime = Tue Mar 31 15:02:20 CEST 2015
> mZxid = 0x1000016bb7
> mtime = Fri Jun 26 22:28:40 CEST 2015
> pZxid = 0xa0006412c
> cversion = 0
> dataVersion = 2
> aclVersion = 0
> ephemeralOwner = 0x0
> dataLength = 53
> numChildren = 0
>
> 2. I tried to overwrite retention.ms once again for my topic. Didn't help.
>
> 3. I looked into logs of the broker and found that it indeed deletes the
> data, but it doesn't print *why* (based on what rule) it deletes the data:
> [2015-06-29 07:35:40,861] INFO Deleting segment 89226232 from log
> my_topic-1. (kafka.log.Log)
> [2015-06-29 07:35:40,993] INFO Deleting index
> /var/lib/kafka/kafka-logs-1/my_topic-1/00000000000089226232.index.deleted (k
> afka.log.OffsetIndex)
>
>
> Please help me, I have no idea what to do about it. Any hint on at least
> how to debug a problem would be great!
>
> Cheers,
> Krzysztof Zarzycki
>