[jira] [Created] (KAFKA-4393) Improve invalid/negative TS handling

["Matthias J. Sax (JIRA)" <ji...@apache.org>]

Matthias J. Sax created KAFKA-4393:
--------------------------------------

             Summary: Improve invalid/negative TS handling
                 Key: KAFKA-4393
                 URL: https://issues.apache.org/jira/browse/KAFKA-4393
             Project: Kafka
          Issue Type: Improvement
          Components: streams
            Reporter: Matthias J. Sax
            Assignee: Matthias J. Sax
             Fix For: 0.10.2.0


Currently, Kafka Streams does not handle invalid/negative timestamps returned from the {{TimestampExtractor}} gracefully, but fails with an exception. Input record timestamps determine output record timestamps and {{KafkaProducer}} does not allow negative timestamps to get written.

Besides the exception issues describe above, negative timestamp can also not get handled in a meaningful way for any time based (ie, window) operators like window aggregates and joins.

Thus, we want to change Stream to a auto-drop behavior for negative timestamps and not process those records at all (without any further user notification about dropped record). To guard the user from silently dropping messages, we change the default extractor {{ConsumerRecordTimestampExtractor}} to check the extracted meta-data record timestamp and raise an exception if it is negative. Furthermore, we add a "drop-and-log" extractor, as this seems to be a common behavior user might want to have. For any other behavior, users can still provide a custom TS-Extractor implementation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)