You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Antony A <an...@gmail.com> on 2018/07/20 17:31:07 UTC
Question - Partition Lag
Hello users.
We have a 7 broker cluster running 0.11.0.2.
We are seeing an issue with one of the partitions in a topic. We see lag on
that partition even though there are many consumer threads available to
process the messages. Any inputs/thoughts?
Here is the metadata
Topic:topic_name PartitionCount:100 ReplicationFactor:1 Configs:
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET
LAG CONSUMER-ID HOST
CLIENT-ID
topic_name 88 2875
2926 51 consumer-192-dea53
xxxxx consumer-192
Thanks,
Antony
Re: Question - Partition Lag
Posted by Antony A <an...@gmail.com>.
I added logging and don’t see any messages consuming. I see lot other consumer waiting for messages, but the broker keep blocking messages only on one session thread
Sent from my mobile. Please excuse any typos.
> On Jul 21, 2018, at 3:08 PM, Liam Clarke <li...@adscale.co.nz> wrote:
>
> Presumably your consumers are blocking or taking a very long time to
> process consumed records. I'd suggest implementing indepth logging of your
> consumers to investigate their performance.
>
>> On Sat, 21 Jul. 2018, 11:51 am Antony A, <an...@gmail.com> wrote:
>>
>> Hello users.
>>
>>
>> We have a 7 broker cluster running 0.11.0.2.
>>
>>
>> We are seeing an issue with one of the partitions in a topic. We see lag on
>> that partition even though there are many consumer threads available to
>> process the messages. Any inputs/thoughts?
>>
>>
>> Here is the metadata
>> Topic:topic_name PartitionCount:100 ReplicationFactor:1 Configs:
>>
>>
>> TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET
>> LAG CONSUMER-ID HOST
>> CLIENT-ID
>> topic_name 88 2875
>> 2926 51 consumer-192-dea53
>> xxxxx consumer-192
>>
>> Thanks,
>> Antony
>>
Re: Question - Partition Lag
Posted by Liam Clarke <li...@adscale.co.nz>.
Presumably your consumers are blocking or taking a very long time to
process consumed records. I'd suggest implementing indepth logging of your
consumers to investigate their performance.
On Sat, 21 Jul. 2018, 11:51 am Antony A, <an...@gmail.com> wrote:
> Hello users.
>
>
> We have a 7 broker cluster running 0.11.0.2.
>
>
> We are seeing an issue with one of the partitions in a topic. We see lag on
> that partition even though there are many consumer threads available to
> process the messages. Any inputs/thoughts?
>
>
> Here is the metadata
> Topic:topic_name PartitionCount:100 ReplicationFactor:1 Configs:
>
>
> TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET
> LAG CONSUMER-ID HOST
> CLIENT-ID
> topic_name 88 2875
> 2926 51 consumer-192-dea53
> xxxxx consumer-192
>
> Thanks,
> Antony
>