You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Simon Cooper <si...@featurespace.co.uk> on 2016/02/05 11:02:13 UTC
0.9 consumer log spam - Marking the coordinator dead
Hi,
We've just updated to the 0.9 client & broker, and we're suddenly seeing a lot of log spam in the consumers:
2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator 2147483646 dead.
2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator 2147483646 dead.
2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator 2147483646 dead.
2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator 2147483646 dead.
2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator 2147483646 dead.
2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator 2147483646 dead.
This is repeated constantly, roughly a whole bunch of messages every 10 minutes. Otherwise the system seems to be operating fine.
Can anyone explain what these messages mean, and what, if anything, we need to do about it?
Thanks,
SimonC
Re: FW: 0.9 consumer log spam - Marking the coordinator dead
Posted by Rajiv Kurian <ra...@signalfx.com>.
Thanks for the update Ismael.
On Fri, Feb 5, 2016 at 10:31 AM, Ismael Juma <is...@juma.me.uk> wrote:
> Hi Rajiv,
>
> Jun just sent a message about 0.9.0.1. It should be out soon if everything
> goes well.
>
> Ismael
>
> On Fri, Feb 5, 2016 at 5:48 PM, Rajiv Kurian <ra...@signalfx.com> wrote:
>
> > Hi Ismael,
> >
> > Is there a maven release planned soon? We've seen this problem too and it
> > is rather disconcerting.
> >
> > Thanks,
> > Rajiv
> >
> > On Fri, Feb 5, 2016 at 5:15 AM, Ismael Juma <is...@juma.me.uk> wrote:
> >
> > > Hi Simon,
> > >
> > > It may be worth trying the 0.9.0 branch as it includes a number of
> > > important fixes to the new consumer.
> > >
> > > Ismael
> > >
> > > On Fri, Feb 5, 2016 at 12:33 PM, Simon Cooper <
> > > simon.cooper@featurespace.co.uk> wrote:
> > >
> > > > Actually, this is incorrect - it looks like the consumer does not
> > receive
> > > > any messages until the *first* 'coordinator dead' message is logged!
> > > >
> > > > Is anyone able to offer insight into what's going on here?
> > > >
> > > > Simon
> > > >
> > > > -----Original Message-----
> > > > From: Simon Cooper [mailto:simon.cooper@featurespace.co.uk]
> > > > Sent: 05 February 2016 10:02
> > > > To: users@kafka.apache.org
> > > > Subject: 0.9 consumer log spam - Marking the coordinator dead
> > > >
> > > > Hi,
> > > >
> > > > We've just updated to the 0.9 client & broker, and we're suddenly
> > seeing
> > > a
> > > > lot of log spam in the consumers:
> > > >
> > > > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > > > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > > > 2147483646 dead.
> > > > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > > > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > > > 2147483646 dead.
> > > > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > > > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > > > 2147483646 dead.
> > > > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > > > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > > > 2147483646 dead.
> > > > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > > > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > > > 2147483646 dead.
> > > > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > > > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > > > 2147483646 dead.
> > > >
> > > > This is repeated constantly, roughly a whole bunch of messages every
> 10
> > > > minutes. Otherwise the system seems to be operating fine.
> > > >
> > > > Can anyone explain what these messages mean, and what, if anything,
> we
> > > > need to do about it?
> > > >
> > > > Thanks,
> > > > SimonC
> > > >
> > >
> >
>
Re: FW: 0.9 consumer log spam - Marking the coordinator dead
Posted by Ismael Juma <is...@juma.me.uk>.
Hi Rajiv,
Jun just sent a message about 0.9.0.1. It should be out soon if everything
goes well.
Ismael
On Fri, Feb 5, 2016 at 5:48 PM, Rajiv Kurian <ra...@signalfx.com> wrote:
> Hi Ismael,
>
> Is there a maven release planned soon? We've seen this problem too and it
> is rather disconcerting.
>
> Thanks,
> Rajiv
>
> On Fri, Feb 5, 2016 at 5:15 AM, Ismael Juma <is...@juma.me.uk> wrote:
>
> > Hi Simon,
> >
> > It may be worth trying the 0.9.0 branch as it includes a number of
> > important fixes to the new consumer.
> >
> > Ismael
> >
> > On Fri, Feb 5, 2016 at 12:33 PM, Simon Cooper <
> > simon.cooper@featurespace.co.uk> wrote:
> >
> > > Actually, this is incorrect - it looks like the consumer does not
> receive
> > > any messages until the *first* 'coordinator dead' message is logged!
> > >
> > > Is anyone able to offer insight into what's going on here?
> > >
> > > Simon
> > >
> > > -----Original Message-----
> > > From: Simon Cooper [mailto:simon.cooper@featurespace.co.uk]
> > > Sent: 05 February 2016 10:02
> > > To: users@kafka.apache.org
> > > Subject: 0.9 consumer log spam - Marking the coordinator dead
> > >
> > > Hi,
> > >
> > > We've just updated to the 0.9 client & broker, and we're suddenly
> seeing
> > a
> > > lot of log spam in the consumers:
> > >
> > > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > > 2147483646 dead.
> > > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > > 2147483646 dead.
> > > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > > 2147483646 dead.
> > > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > > 2147483646 dead.
> > > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > > 2147483646 dead.
> > > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > > 2147483646 dead.
> > >
> > > This is repeated constantly, roughly a whole bunch of messages every 10
> > > minutes. Otherwise the system seems to be operating fine.
> > >
> > > Can anyone explain what these messages mean, and what, if anything, we
> > > need to do about it?
> > >
> > > Thanks,
> > > SimonC
> > >
> >
>
Re: FW: 0.9 consumer log spam - Marking the coordinator dead
Posted by Rajiv Kurian <ra...@signalfx.com>.
Hi Ismael,
Is there a maven release planned soon? We've seen this problem too and it
is rather disconcerting.
Thanks,
Rajiv
On Fri, Feb 5, 2016 at 5:15 AM, Ismael Juma <is...@juma.me.uk> wrote:
> Hi Simon,
>
> It may be worth trying the 0.9.0 branch as it includes a number of
> important fixes to the new consumer.
>
> Ismael
>
> On Fri, Feb 5, 2016 at 12:33 PM, Simon Cooper <
> simon.cooper@featurespace.co.uk> wrote:
>
> > Actually, this is incorrect - it looks like the consumer does not receive
> > any messages until the *first* 'coordinator dead' message is logged!
> >
> > Is anyone able to offer insight into what's going on here?
> >
> > Simon
> >
> > -----Original Message-----
> > From: Simon Cooper [mailto:simon.cooper@featurespace.co.uk]
> > Sent: 05 February 2016 10:02
> > To: users@kafka.apache.org
> > Subject: 0.9 consumer log spam - Marking the coordinator dead
> >
> > Hi,
> >
> > We've just updated to the 0.9 client & broker, and we're suddenly seeing
> a
> > lot of log spam in the consumers:
> >
> > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > 2147483646 dead.
> > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > 2147483646 dead.
> > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > 2147483646 dead.
> > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > 2147483646 dead.
> > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > 2147483646 dead.
> > 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> > [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> > 2147483646 dead.
> >
> > This is repeated constantly, roughly a whole bunch of messages every 10
> > minutes. Otherwise the system seems to be operating fine.
> >
> > Can anyone explain what these messages mean, and what, if anything, we
> > need to do about it?
> >
> > Thanks,
> > SimonC
> >
>
RE: FW: 0.9 consumer log spam - Marking the coordinator dead
Posted by Simon Cooper <si...@featurespace.co.uk>.
Thanks, I'll have a look - is there a 0.9.0.1 release planned soon?
Simon
-----Original Message-----
From: ismaelj@gmail.com [mailto:ismaelj@gmail.com] On Behalf Of Ismael Juma
Sent: 05 February 2016 13:15
To: users@kafka.apache.org
Subject: Re: FW: 0.9 consumer log spam - Marking the coordinator dead
Hi Simon,
It may be worth trying the 0.9.0 branch as it includes a number of important fixes to the new consumer.
Ismael
On Fri, Feb 5, 2016 at 12:33 PM, Simon Cooper < simon.cooper@featurespace.co.uk> wrote:
> Actually, this is incorrect - it looks like the consumer does not
> receive any messages until the *first* 'coordinator dead' message is logged!
>
> Is anyone able to offer insight into what's going on here?
>
> Simon
>
> -----Original Message-----
> From: Simon Cooper [mailto:simon.cooper@featurespace.co.uk]
> Sent: 05 February 2016 10:02
> To: users@kafka.apache.org
> Subject: 0.9 consumer log spam - Marking the coordinator dead
>
> Hi,
>
> We've just updated to the 0.9 client & broker, and we're suddenly
> seeing a lot of log spam in the consumers:
>
> 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> 2147483646 dead.
> 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> 2147483646 dead.
> 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> 2147483646 dead.
> 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> 2147483646 dead.
> 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> 2147483646 dead.
> 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> 2147483646 dead.
>
> This is repeated constantly, roughly a whole bunch of messages every
> 10 minutes. Otherwise the system seems to be operating fine.
>
> Can anyone explain what these messages mean, and what, if anything, we
> need to do about it?
>
> Thanks,
> SimonC
>
Re: FW: 0.9 consumer log spam - Marking the coordinator dead
Posted by Ismael Juma <is...@juma.me.uk>.
Hi Simon,
It may be worth trying the 0.9.0 branch as it includes a number of
important fixes to the new consumer.
Ismael
On Fri, Feb 5, 2016 at 12:33 PM, Simon Cooper <
simon.cooper@featurespace.co.uk> wrote:
> Actually, this is incorrect - it looks like the consumer does not receive
> any messages until the *first* 'coordinator dead' message is logged!
>
> Is anyone able to offer insight into what's going on here?
>
> Simon
>
> -----Original Message-----
> From: Simon Cooper [mailto:simon.cooper@featurespace.co.uk]
> Sent: 05 February 2016 10:02
> To: users@kafka.apache.org
> Subject: 0.9 consumer log spam - Marking the coordinator dead
>
> Hi,
>
> We've just updated to the 0.9 client & broker, and we're suddenly seeing a
> lot of log spam in the consumers:
>
> 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> 2147483646 dead.
> 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> 2147483646 dead.
> 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> 2147483646 dead.
> 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> 2147483646 dead.
> 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> 2147483646 dead.
> 2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator
> [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator
> 2147483646 dead.
>
> This is repeated constantly, roughly a whole bunch of messages every 10
> minutes. Otherwise the system seems to be operating fine.
>
> Can anyone explain what these messages mean, and what, if anything, we
> need to do about it?
>
> Thanks,
> SimonC
>
FW: 0.9 consumer log spam - Marking the coordinator dead
Posted by Simon Cooper <si...@featurespace.co.uk>.
Actually, this is incorrect - it looks like the consumer does not receive any messages until the *first* 'coordinator dead' message is logged!
Is anyone able to offer insight into what's going on here?
Simon
-----Original Message-----
From: Simon Cooper [mailto:simon.cooper@featurespace.co.uk]
Sent: 05 February 2016 10:02
To: users@kafka.apache.org
Subject: 0.9 consumer log spam - Marking the coordinator dead
Hi,
We've just updated to the 0.9 client & broker, and we're suddenly seeing a lot of log spam in the consumers:
2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator 2147483646 dead.
2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator 2147483646 dead.
2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator 2147483646 dead.
2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator 2147483646 dead.
2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator 2147483646 dead.
2016-02-05T03:31:34,182 | INFO | o.a.k.c.c.i.AbstractCoordinator [kafkaspout-thread-0] | Response-events:7 | Marking the coordinator 2147483646 dead.
This is repeated constantly, roughly a whole bunch of messages every 10 minutes. Otherwise the system seems to be operating fine.
Can anyone explain what these messages mean, and what, if anything, we need to do about it?
Thanks,
SimonC