Re: log.message.timestamp.difference.max.ms and future timestamps?

["Matthias J. Sax" <mj...@apache.org>]

>> What happens if the message timestamp is in the future?

If the difference if larger than
`log.message.timestamp.difference.max.ms` the write will be rejected.

This timestamp difference works both ways.


-Matthias


On 4/16/20 9:39 AM, Andrew Otto wrote:
> log.message.timestamp.difference.max.ms is the "The maximum difference
> allowed between the timestamp when a broker receives a message and the
> timestamp specified in the message.".
> 
> What happens if the message timestamp is in the future?  We just
> encountered a problem where a producer set a timestamp a year in the
> future, which is keeping log segments with that message to not be deleted,
> causing disks to fill.  I'd like to reject messages with timestamps (too
> far) in the future.  Based on the way the docs are worded, I'd assume the
> difference will be negative in this case.
> 
>   log.message.timestamp.difference.max.ms - futureTimestamp == -bigNumber
> 
> Will the message be rejected or accepted in this case?
> 
> Thanks!
> -Andrew Otto
>  @Wikimedia Foundation
> 

Re: log.message.timestamp.difference.max.ms and future timestamps?

[Andrew Otto <ot...@wikimedia.org>]

Thank you, good to know!

On Fri, May 8, 2020 at 2:53 PM Matthias J. Sax <mj...@apache.org> wrote:

> >> What happens if the message timestamp is in the future?
>
> If the difference if larger than
> `log.message.timestamp.difference.max.ms` the write will be rejected.
>
> This timestamp difference works both ways.
>
>
> -Matthias
>
>
> On 4/16/20 9:39 AM, Andrew Otto wrote:
> > log.message.timestamp.difference.max.ms is the "The maximum difference
> > allowed between the timestamp when a broker receives a message and the
> > timestamp specified in the message.".
> >
> > What happens if the message timestamp is in the future?  We just
> > encountered a problem where a producer set a timestamp a year in the
> > future, which is keeping log segments with that message to not be
> deleted,
> > causing disks to fill.  I'd like to reject messages with timestamps (too
> > far) in the future.  Based on the way the docs are worded, I'd assume the
> > difference will be negative in this case.
> >
> >   log.message.timestamp.difference.max.ms - futureTimestamp ==
> -bigNumber
> >
> > Will the message be rejected or accepted in this case?
> >
> > Thanks!
> > -Andrew Otto
> >  @Wikimedia Foundation
> >
>
>