[jira] [Created] (KAFKA-13818) Add generation to consumer assignor logs

["John Roesler (Jira)" <ji...@apache.org>]

John Roesler created KAFKA-13818:
------------------------------------

             Summary: Add generation to consumer assignor logs
                 Key: KAFKA-13818
                 URL: https://issues.apache.org/jira/browse/KAFKA-13818
             Project: Kafka
          Issue Type: Improvement
            Reporter: John Roesler


Reading assignor logs is really confusing in large part because they are
spread across different layers of abstraction (the ConsumerCoordinator
and the ConsumerPartitionAssignor, which in Streams consists of several
layers of its own). Each layer in the abstraction reports useful information
that only it has access to, but because they are split over multiple lines, with
multiple members in the cluster, and (often) multiple rebalances taking place
in rapid succession, it's often hard to understand which logs are part of
which rebalance.

 

One thing we don't want to do is break encapsulation by exposing too much of the ConsumerCoordinator's internal state to components like the pluggable ConsumerPartitionAssignor.

 

We can accomplish what we want by adding the concept of a dynamic log context, so that the ConsumerCoordinator can add dynamic information like the generation id to be logged for correlation in other components without exposing any new information or metadata to those components themselves.

See [https://github.com/apache/kafka/pull/12020] for example.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)