You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Siva A <si...@gmail.com> on 2018/01/25 06:53:39 UTC

Kafka Consumer Issue

Hi All,

I have a 3 node Kafka cluster.
I am trying to consume data from logstash(version 5.5.2) using the new
consumer API.

When Kafka2 and Kafka3 is down i am still able to consume the data without
any issues.

But whenever the kafka1 is down the logstash consumer is just hang there.
Anyone faced this kind of issue in any consumer(logstash or java or
anything)?

Thanks in advance.

Thanks
Siva

Re: Kafka Consumer Issue

Posted by Siva A <si...@gmail.com>.

Yes it was the issue. Fixed yesterday. Thanks for your update.

On Jan 26, 2018 11:27 AM, "chintan mavawala" <ch...@gmail.com> wrote:

> You may want to check replication factor of _consumer_offsets topic. By
> default, it is 1. It should be increased to 3 in your case.
>
> Regards,
> Chintan
>
> On 25-Jan-2018 12:24 PM, "Siva A" <si...@gmail.com> wrote:
>
> > Kafka version i am using is 0.10.0.1
> >
> > On Thu, Jan 25, 2018 at 12:23 PM, Siva A <si...@gmail.com>
> wrote:
> >
> > > Hi All,
> > >
> > > I have a 3 node Kafka cluster.
> > > I am trying to consume data from logstash(version 5.5.2) using the new
> > > consumer API.
> > >
> > > When Kafka2 and Kafka3 is down i am still able to consume the data
> > without
> > > any issues.
> > >
> > > But whenever the kafka1 is down the logstash consumer is just hang
> there.
> > > Anyone faced this kind of issue in any consumer(logstash or java or
> > > anything)?
> > >
> > > Thanks in advance.
> > >
> > > Thanks
> > > Siva
> > >
> >
>

Re: Kafka Consumer Issue

Posted by chintan mavawala <ch...@gmail.com>.

You may want to check replication factor of _consumer_offsets topic. By
default, it is 1. It should be increased to 3 in your case.

Regards,
Chintan

On 25-Jan-2018 12:24 PM, "Siva A" <si...@gmail.com> wrote:

> Kafka version i am using is 0.10.0.1
>
> On Thu, Jan 25, 2018 at 12:23 PM, Siva A <si...@gmail.com> wrote:
>
> > Hi All,
> >
> > I have a 3 node Kafka cluster.
> > I am trying to consume data from logstash(version 5.5.2) using the new
> > consumer API.
> >
> > When Kafka2 and Kafka3 is down i am still able to consume the data
> without
> > any issues.
> >
> > But whenever the kafka1 is down the logstash consumer is just hang there.
> > Anyone faced this kind of issue in any consumer(logstash or java or
> > anything)?
> >
> > Thanks in advance.
> >
> > Thanks
> > Siva
> >
>

Re: Kafka Consumer Issue

Posted by Siva A <si...@gmail.com>.

Kafka version i am using is 0.10.0.1

On Thu, Jan 25, 2018 at 12:23 PM, Siva A <si...@gmail.com> wrote:

> Hi All,
>
> I have a 3 node Kafka cluster.
> I am trying to consume data from logstash(version 5.5.2) using the new
> consumer API.
>
> When Kafka2 and Kafka3 is down i am still able to consume the data without
> any issues.
>
> But whenever the kafka1 is down the logstash consumer is just hang there.
> Anyone faced this kind of issue in any consumer(logstash or java or
> anything)?
>
> Thanks in advance.
>
> Thanks
> Siva
>