[jira] [Created] (KAFKA-17109) Reduce log message load for failed locking

["Bruno Cadonna (Jira)" <ji...@apache.org>]

Bruno Cadonna created KAFKA-17109:
-------------------------------------

             Summary: Reduce log message load for failed locking
                 Key: KAFKA-17109
                 URL: https://issues.apache.org/jira/browse/KAFKA-17109
             Project: Kafka
          Issue Type: Improvement
          Components: streams
    Affects Versions: 3.8.0
            Reporter: Bruno Cadonna


The following exception with stack traces is logged many times when state updater is enabled:

{code}
01:08:03 INFO  [KAFKA] TaskManager - stream-thread [acme-StreamThread-4] Encountered lock exception. Reattempting locking the state in the next iteration.
org.apache.kafka.streams.errors.LockException: stream-thread [acme-StreamThread-4] standby-task [1_15] Failed to lock the state directory for task 1_15
	at org.apache.kafka.streams.processor.internals.StateManagerUtil.registerStateStores(StateManagerUtil.java:96)
	at org.apache.kafka.streams.processor.internals.StandbyTask.initializeIfNeeded(StandbyTask.java:114)
	at org.apache.kafka.streams.processor.internals.TaskManager.addTaskToStateUpdater(TaskManager.java:1008) 
	at org.apache.kafka.streams.processor.internals.TaskManager.addTasksToStateUpdater(TaskManager.java:995) 
	at org.apache.kafka.streams.processor.internals.TaskManager.checkStateUpdater(TaskManager.java:911) 
	at org.apache.kafka.streams.processor.internals.StreamThread.checkStateUpdater(StreamThread.java:1188) 
	at org.apache.kafka.streams.processor.internals.StreamThread.runOnceWithoutProcessingThreads(StreamThread.java:996) 
	at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:711) 
	at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:670) {code}

The exception is expected since it happens because a lock on the task state directory is not yet been freed by a different stream thread on the same Kafka Streams client after an assignment. But with the state updater acquiring the lock is attempted in each poll iteration which is every 100 ms by default.

One option to reduce the log messages is to reduce the rate at which a lock is attempted to be acquired. The other is to reduce the logging.  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)