[jira] [Commented] (KAFKA-6939) Change the default of log.message.timestamp.difference.max.ms to 500 years

["Badai Aqrandista (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-6939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529386#comment-16529386 ] 

Badai Aqrandista commented on KAFKA-6939:
-----------------------------------------

[~bgummalla]

Yes, this issue still exist in the code because Kafka will assume the timestamp is in millisecond. But if customer incorrectly assume it's in second or microsecond, Kafka will still accept it but the message will be deleted immediately (if timestamp is in second) or generate a lot of segment files (if timestamp is in microsecond).

So, a reasonable default for {{log.message.timestamp.difference.max.ms}} should prevent user from these incorrect assumption. Especially when Kafka client is not Java based, with default time library generating times not in millisecond.

Thanks
Badai

> Change the default of log.message.timestamp.difference.max.ms to 500 years
> --------------------------------------------------------------------------
>
>                 Key: KAFKA-6939
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6939
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Badai Aqrandista
>            Priority: Minor
>
> If producer incorrectly provides timestamp in microsecond (not in millisecond), the record is accepted by default and can cause broker to roll the segment files continuously. And on a heavily used broker, this will generate a lot of index files, which then causes the broker to hit `vm.max_map_count`.
> So I'd like to suggest changing the default for log.message.timestamp.difference.max.ms to 15768000000000 (500 years * 365 days * 86400 seconds * 1000). This would reject timestamp in microsecond from producer and still allow most historical data to be stored in Kafka.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)