You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by "M. Manna" <ma...@gmail.com> on 2017/07/08 22:46:28 UTC

Effects of operation with log.retention.bytes and log.segment.bytes

Hello,

I have been running some stress tests regarding log retention whilst
writing to kafka continuously every 5 seconds.

I have set the following properties (apart from other mandatory ones):
log.retention.minutes=3
log.retention.hours.
log.retention.bytes=26214400
log.segment.bytes=10485760
log.retention.check.interval.ms=240000
offsets.retention.check.interval.ms=300000
offsets.retention.minutes=1414690
log.cleanup.policy=delete

If my understanding is correct, the logs will probably not be deleted since
every segment is under the limit? But the documentation says that the logs
will be deleted if it satisfies either (log.retention.minutes) or
(log.retention.bytes) - but I actually don't see the scheduled cleanup
kicking in roughly every 3 minutes or so.

Is this expected? If so, what should the ideal relationship be between
these retention params?

KR,