Kafka Log deletion Problem

[SenthilKumar K <se...@gmail.com>]

Hello Experts , We have a Kafka Setup running for our analytics pipeline
...Below is the broker config ..

max.message.bytes = 67108864
replica.fetch.max.bytes = 67108864
zookeeper.session.timeout.ms = 7000
replica.socket.timeout.ms = 30000
offsets.commit.timeout.ms = 5000
request.timeout.ms = 40000
zookeeper.connection.timeout.ms = 7000
controller.socket.timeout.ms = 30000
num.partitions = 24
listeners = SSL://23.212.237.10:9093
broker.id = 1
socket.receive.buffer.bytes = 102400
message.max.bytes = 2621440
auto.create.topics.enable = true
auto.leader.rebalance.enable = true
zookeeper.connect = zk1:2181,zk2:2181
log.retention.ms=86400000
#log.retention.hours = 24
socket.request.max.bytes = 104857600
default.replication.factor = 2
log.dirs = /data/kafka_logs
compression.codec = 3

Kafka Version : *0.11*

The retention period is set to 24 Hours , but i could see the data in disk
after 24 hours.. *What could be the problem here ?*

Note : There is no topic specific configuration.

--Senthil

Re: Kafka Log deletion Problem

[Manikumar <ma...@gmail.com>]

looks like log segment is not rotated.  You can send more data or adjust
log.roll.ms/log.segment.bytes configs to rotate segments.


On Fri, Feb 2, 2018 at 7:08 PM, SenthilKumar K <se...@gmail.com>
wrote:

> Hello Experts , We have a Kafka Setup running for our analytics pipeline
> ...Below is the broker config ..
>
> max.message.bytes = 67108864
> replica.fetch.max.bytes = 67108864
> zookeeper.session.timeout.ms = 7000
> replica.socket.timeout.ms = 30000
> offsets.commit.timeout.ms = 5000
> request.timeout.ms = 40000
> zookeeper.connection.timeout.ms = 7000
> controller.socket.timeout.ms = 30000
> num.partitions = 24
> listeners = SSL://23.212.237.10:9093
> broker.id = 1
> socket.receive.buffer.bytes = 102400
> message.max.bytes = 2621440
> auto.create.topics.enable = true
> auto.leader.rebalance.enable = true
> zookeeper.connect = zk1:2181,zk2:2181
> log.retention.ms=86400000
> #log.retention.hours = 24
> socket.request.max.bytes = 104857600
> default.replication.factor = 2
> log.dirs = /data/kafka_logs
> compression.codec = 3
>
> Kafka Version : *0.11*
>
> The retention period is set to 24 Hours , but i could see the data in disk
> after 24 hours.. *What could be the problem here ?*
>
> Note : There is no topic specific configuration.
>
> --Senthil
>