Understanding Kafka controller log

[Amitav Mohanty <am...@gmail.com>]

Hi

I am trying to understand the right way of viewing controller
 logs. As the controller logs are written in each broker, do we need
 to see all of them to know the state of the cluster at any given
 point in time ? To follow the leader of any given partition, do we need to
correlate
across controller logs of all brokers ?

Regards,
Amitav

Re: Understanding Kafka controller log

[Amitav Mohanty <am...@gmail.com>]

Hi Liam,

Thank you for the clarification.

My use of words was a bit confusing. Let me rephrase it. :)

I believe each partition has a leader. This is liable to change in case of
any broker
going down. What I am interested in it getting logs from the current leader
of any partition. For
that, I will have to get logs from all brokers and then stitch then based
on partition leader to get
a timeline of logs. Does that make sense?

Regards,
Amitav

On Wed, May 20, 2020 at 4:52 AM Liam Clarke-Hutchinson <
liam.clarke@adscale.co.nz> wrote:

> Hi Amitav,
>
> Brokers only write to controller log when they're the cluster controller.
> So if you wanted to see what the cluster controller was doing at a given
> point in time, then yep, you'd want to aggregate all controller.log files
> to handle controller changes. Typically controller status is very stable,
> it only really changes if you're restarting a broker or it fails.
>
> You can tell which broker is controller by looking at the JMX exposed
> metric kafka.controller:type=KafkaController,name=ActiveControllerCount,
> it'll be 1 if a broker is the controller, 0 otherwise (and if that metric
> ever adds up to more than 1 across your cluster, you've got problems).
>
> > To follow the leader of any given partition, do we need to correlate
> across controller logs of all brokers ?
>
> What do you mean by follow the leader?
>
> Cheers,
>
> Liam Clarke-Hutchinson
>
>
> On Wed, 20 May 2020, 7:04 am Amitav Mohanty, <am...@gmail.com>
> wrote:
>
> > Hi
> >
> > I am trying to understand the right way of viewing controller
> >  logs. As the controller logs are written in each broker, do we need
> >  to see all of them to know the state of the cluster at any given
> >  point in time ? To follow the leader of any given partition, do we need
> to
> > correlate
> > across controller logs of all brokers ?
> >
> > Regards,
> > Amitav
> >
>

Re: Understanding Kafka controller log

[Liam Clarke-Hutchinson <li...@adscale.co.nz>]

Hi Amitav,

Brokers only write to controller log when they're the cluster controller.
So if you wanted to see what the cluster controller was doing at a given
point in time, then yep, you'd want to aggregate all controller.log files
to handle controller changes. Typically controller status is very stable,
it only really changes if you're restarting a broker or it fails.

You can tell which broker is controller by looking at the JMX exposed
metric kafka.controller:type=KafkaController,name=ActiveControllerCount,
it'll be 1 if a broker is the controller, 0 otherwise (and if that metric
ever adds up to more than 1 across your cluster, you've got problems).

> To follow the leader of any given partition, do we need to correlate
across controller logs of all brokers ?

What do you mean by follow the leader?

Cheers,

Liam Clarke-Hutchinson


On Wed, 20 May 2020, 7:04 am Amitav Mohanty, <am...@gmail.com>
wrote:

> Hi
>
> I am trying to understand the right way of viewing controller
>  logs. As the controller logs are written in each broker, do we need
>  to see all of them to know the state of the cluster at any given
>  point in time ? To follow the leader of any given partition, do we need to
> correlate
> across controller logs of all brokers ?
>
> Regards,
> Amitav
>