You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Konstantine Karantasis (Jira)" <ji...@apache.org> on 2021/07/09 03:44:00 UTC
[jira] [Commented] (KAFKA-12176) Consider changing default
log.message.timestamp.difference.max.ms
[ https://issues.apache.org/jira/browse/KAFKA-12176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17377771#comment-17377771 ]
Konstantine Karantasis commented on KAFKA-12176:
------------------------------------------------
[~hachikuji] [~junrao] I don't see any evidence that this feature made into 3.0 before the KIP and Feature freezes respectively. I'm moving the target release to the next one, but let me know if I've missed something regarding this issue.
> Consider changing default log.message.timestamp.difference.max.ms
> -----------------------------------------------------------------
>
> Key: KAFKA-12176
> URL: https://issues.apache.org/jira/browse/KAFKA-12176
> Project: Kafka
> Issue Type: Improvement
> Reporter: Jason Gustafson
> Priority: Major
> Labels: needs-kip
> Fix For: 3.0.0
>
>
> The default `log.message.timestamp.difference.max.ms` is Long.MaxValue, which means the broker will accept arbitrary timestamps. The broker relies on timestamps internally for deciding when a segments should be rolled and when they should be deleted. A buggy client which is producing messages with timestamps too far in the future or past can cause segments to roll frequently which can lead to open file exhaustion, or it can cause segments to be retained indefinitely which can lead to disk space exhaustion.
> In https://issues.apache.org/jira/browse/KAFKA-4340, it was proposed to set the default equal to `log.retention.ms`, which still seems to make sense. This was rejected for two reasons as far as I can tell. First was compatibility, which just means we would need a major upgrade. The second is that we previously did not have a way to tell the user which record had violated the max timestamp difference, but now we have [KIP-467|https://cwiki.apache.org/confluence/display/KAFKA/KIP-467%3A+Augment+ProduceResponse+error+messaging+for+specific+culprit+records].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)