[jira] [Created] (KAFKA-8251) Consider different logging for purgeData

["Matthias J. Sax (JIRA)" <ji...@apache.org>]

Matthias J. Sax created KAFKA-8251:
--------------------------------------

             Summary: Consider different logging for purgeData
                 Key: KAFKA-8251
                 URL: https://issues.apache.org/jira/browse/KAFKA-8251
             Project: Kafka
          Issue Type: Improvement
          Components: core
            Reporter: Matthias J. Sax


As reported in this SO question [https://stackoverflow.com/questions/55724259/kafka-streams-floods-kafka-logs/55738257#55738257] each time data is purged from a topic a log message is written:

> {{[2019-04-17 09:06:16,541] INFO [Log partition=my-application-KSTREAM-AGGREGATE-STATE-STORE-0000000076-repartition-0, dir=/opt/kafka/data/logs] Incrementing log start offset to 316423 (kafka.log.Log) [2019-04-17 09:06:16,545] INFO [Log partition=my-application-KSTREAM-AGGREGATE-STATE-STORE-0000000033-repartition-2, dir=/opt/kafka/data/logs] Incrementing log start offset to 3394 (kafka.log.Log) }}

While this is useful for regular log truncation, it seems less useful for explicit purge-data requests. Kafka Streams uses the purge data API on repartition topics aggressively and this can result to verbose logging.

I don't know this part of the code, but maybe it's possible to distinguish both cases and only log on time/size based truncation at INFO level and reduce log level for purge-data request to TRACE?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)