You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Jose Raul Perez Rodriguez <jo...@gmail.com> on 2016/04/26 14:19:57 UTC

GroupCoordinatorNotAvailableException, consumer error on poll request

Hello,

I am using KafkaProducer and KafkaConsumer, version 0.90, in java, the 
producer works good (the Console Consumer provided in samples receives 
the producer data).
But in the configured KafkaConsumer I got the 
/org.apache.kafka.common.errors.GroupCoordinatorNotAvailableException/**error
The error is thrown in /handleGroupMetadataResponse(ClientResponse resp, 
RequestFuture<Void> future)/**method**of class 
/org.apache.kafka.clients.consumer.internals.AbstractCoordinator/
The consumer configuration:

/props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,"0.1.2.3:9092");//
//props.put(ConsumerConfig.GROUP_ID_CONFIG, "my_group");//
//props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "true");//
//props.put(ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG, "60000");//
//props.put(ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG, "20000");//
//props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, 
"org.apache.kafka.common.serialization.IntegerDeserializer");//
//props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, 
"org.apache.kafka.common.serialization.StringDeserializer");//

/The broker configuration has no significant changes.
I detect the error using Kafka from source code and tracing the methods 
manually, because no raise any exception when used from compiled .jars.
What can i do in order to, at least, known more details about this 
error, or ideally solving it ;-)

Thanks

Re: GroupCoordinatorNotAvailableException, consumer error on poll request

Posted by Jose Raul Perez Rodriguez <jo...@gmail.com>.
Problem solved!

I had to reconfigure some brokers, to cleanup the configuration and 
metadata in __consumer offset.

Thanks Jason


On 26/04/16 23:06, Jason Gustafson wrote:
> Hey Jose,
>
> That error could indicate that one of the partitions of the
> __consumer_offsets topic has no leader. The group coordinators are chosen
> from the leaders of those partitions, so if one is unavailable, then the
> consumer won't be able to find the group. Can you show the output of the
> following command:
>
> bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic
> __consumer_offsets
>
> (where the zookeeper hostname is replaced accordingly).
>
> -Jason
>
> On Tue, Apr 26, 2016 at 1:08 PM, José Raúl Pérez Rodríguez <
> joseraul.work@gmail.com> wrote:
>
>> That is the problem,
>>
>> I can't see the stack trace, the KafkaConsumer don't throw any Exception,
>> it simply stay in *poll() *method and never return. I  realized the
>> internal error was GroupCoordinatorNotAvailableException by using Kafka
>> from source code and tracing the method calls, I saw that *poll() *never
>> return because entered in a while loop whose condition depended on a
>> *Future*<> object that always fails, and the fail cause is
>> GroupCoordinatorNotAvailableException.
>>
>> Thans
>>
>>
>>
>>
>> On 4/26/2016 8:50 PM, Jason Gustafson wrote:
>>
>>> Hi Jose,
>>>
>>> Can you provide the full exception stack trace? Which method is this being
>>> throw from? It might help to see the snippet which is producing it as
>>> well.
>>>
>>> Thanks,
>>> Jason
>>>
>>> On Tue, Apr 26, 2016 at 5:19 AM, Jose Raul Perez Rodriguez <
>>> joseraul.work@gmail.com> wrote:
>>>
>>> Hello,
>>>> I am using KafkaProducer and KafkaConsumer, version 0.90, in java, the
>>>> producer works good (the Console Consumer provided in samples receives
>>>> the
>>>> producer data).
>>>> But in the configured KafkaConsumer I got the
>>>>
>>>> /org.apache.kafka.common.errors.GroupCoordinatorNotAvailableException/**error
>>>> The error is thrown in /handleGroupMetadataResponse(ClientResponse resp,
>>>> RequestFuture<Void> future)/**method**of class
>>>> /org.apache.kafka.clients.consumer.internals.AbstractCoordinator/
>>>> The consumer configuration:
>>>>
>>>> /props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,"0.1.2.3:9092");//
>>>> //props.put(ConsumerConfig.GROUP_ID_CONFIG, "my_group");//
>>>> //props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "true");//
>>>> //props.put(ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG, "60000");//
>>>> //props.put(ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG, "20000");//
>>>> //props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG,
>>>> "org.apache.kafka.common.serialization.IntegerDeserializer");//
>>>> //props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,
>>>> "org.apache.kafka.common.serialization.StringDeserializer");//
>>>>
>>>> /The broker configuration has no significant changes.
>>>> I detect the error using Kafka from source code and tracing the methods
>>>> manually, because no raise any exception when used from compiled .jars.
>>>> What can i do in order to, at least, known more details about this error,
>>>> or ideally solving it ;-)
>>>>
>>>> Thanks
>>>>
>>>>


Re: GroupCoordinatorNotAvailableException, consumer error on poll request

Posted by Jason Gustafson <ja...@confluent.io>.
Hey Jose,

That error could indicate that one of the partitions of the
__consumer_offsets topic has no leader. The group coordinators are chosen
from the leaders of those partitions, so if one is unavailable, then the
consumer won't be able to find the group. Can you show the output of the
following command:

bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic
__consumer_offsets

(where the zookeeper hostname is replaced accordingly).

-Jason

On Tue, Apr 26, 2016 at 1:08 PM, José Raúl Pérez Rodríguez <
joseraul.work@gmail.com> wrote:

> That is the problem,
>
> I can't see the stack trace, the KafkaConsumer don't throw any Exception,
> it simply stay in *poll() *method and never return. I  realized the
> internal error was GroupCoordinatorNotAvailableException by using Kafka
> from source code and tracing the method calls, I saw that *poll() *never
> return because entered in a while loop whose condition depended on a
> *Future*<> object that always fails, and the fail cause is
> GroupCoordinatorNotAvailableException.
>
> Thans
>
>
>
>
> On 4/26/2016 8:50 PM, Jason Gustafson wrote:
>
>> Hi Jose,
>>
>> Can you provide the full exception stack trace? Which method is this being
>> throw from? It might help to see the snippet which is producing it as
>> well.
>>
>> Thanks,
>> Jason
>>
>> On Tue, Apr 26, 2016 at 5:19 AM, Jose Raul Perez Rodriguez <
>> joseraul.work@gmail.com> wrote:
>>
>> Hello,
>>>
>>> I am using KafkaProducer and KafkaConsumer, version 0.90, in java, the
>>> producer works good (the Console Consumer provided in samples receives
>>> the
>>> producer data).
>>> But in the configured KafkaConsumer I got the
>>>
>>> /org.apache.kafka.common.errors.GroupCoordinatorNotAvailableException/**error
>>> The error is thrown in /handleGroupMetadataResponse(ClientResponse resp,
>>> RequestFuture<Void> future)/**method**of class
>>> /org.apache.kafka.clients.consumer.internals.AbstractCoordinator/
>>> The consumer configuration:
>>>
>>> /props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,"0.1.2.3:9092");//
>>> //props.put(ConsumerConfig.GROUP_ID_CONFIG, "my_group");//
>>> //props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "true");//
>>> //props.put(ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG, "60000");//
>>> //props.put(ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG, "20000");//
>>> //props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG,
>>> "org.apache.kafka.common.serialization.IntegerDeserializer");//
>>> //props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,
>>> "org.apache.kafka.common.serialization.StringDeserializer");//
>>>
>>> /The broker configuration has no significant changes.
>>> I detect the error using Kafka from source code and tracing the methods
>>> manually, because no raise any exception when used from compiled .jars.
>>> What can i do in order to, at least, known more details about this error,
>>> or ideally solving it ;-)
>>>
>>> Thanks
>>>
>>>
>

Re: GroupCoordinatorNotAvailableException, consumer error on poll request

Posted by José Raúl Pérez Rodríguez <jo...@gmail.com>.
That is the problem,

I can't see the stack trace, the KafkaConsumer don't throw any 
Exception, it simply stay in *poll() *method and never return. I  
realized the internal error was GroupCoordinatorNotAvailableException by 
using Kafka from source code and tracing the method calls, I saw that 
*poll() *never return because entered in a while loop whose condition 
depended on a *Future*<> object that always fails, and the fail cause is 
GroupCoordinatorNotAvailableException.

Thans



On 4/26/2016 8:50 PM, Jason Gustafson wrote:
> Hi Jose,
>
> Can you provide the full exception stack trace? Which method is this being
> throw from? It might help to see the snippet which is producing it as well.
>
> Thanks,
> Jason
>
> On Tue, Apr 26, 2016 at 5:19 AM, Jose Raul Perez Rodriguez <
> joseraul.work@gmail.com> wrote:
>
>> Hello,
>>
>> I am using KafkaProducer and KafkaConsumer, version 0.90, in java, the
>> producer works good (the Console Consumer provided in samples receives the
>> producer data).
>> But in the configured KafkaConsumer I got the
>> /org.apache.kafka.common.errors.GroupCoordinatorNotAvailableException/**error
>> The error is thrown in /handleGroupMetadataResponse(ClientResponse resp,
>> RequestFuture<Void> future)/**method**of class
>> /org.apache.kafka.clients.consumer.internals.AbstractCoordinator/
>> The consumer configuration:
>>
>> /props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,"0.1.2.3:9092");//
>> //props.put(ConsumerConfig.GROUP_ID_CONFIG, "my_group");//
>> //props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "true");//
>> //props.put(ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG, "60000");//
>> //props.put(ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG, "20000");//
>> //props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG,
>> "org.apache.kafka.common.serialization.IntegerDeserializer");//
>> //props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,
>> "org.apache.kafka.common.serialization.StringDeserializer");//
>>
>> /The broker configuration has no significant changes.
>> I detect the error using Kafka from source code and tracing the methods
>> manually, because no raise any exception when used from compiled .jars.
>> What can i do in order to, at least, known more details about this error,
>> or ideally solving it ;-)
>>
>> Thanks
>>


Re: GroupCoordinatorNotAvailableException, consumer error on poll request

Posted by Jason Gustafson <ja...@confluent.io>.
Hi Jose,

Can you provide the full exception stack trace? Which method is this being
throw from? It might help to see the snippet which is producing it as well.

Thanks,
Jason

On Tue, Apr 26, 2016 at 5:19 AM, Jose Raul Perez Rodriguez <
joseraul.work@gmail.com> wrote:

> Hello,
>
> I am using KafkaProducer and KafkaConsumer, version 0.90, in java, the
> producer works good (the Console Consumer provided in samples receives the
> producer data).
> But in the configured KafkaConsumer I got the
> /org.apache.kafka.common.errors.GroupCoordinatorNotAvailableException/**error
> The error is thrown in /handleGroupMetadataResponse(ClientResponse resp,
> RequestFuture<Void> future)/**method**of class
> /org.apache.kafka.clients.consumer.internals.AbstractCoordinator/
> The consumer configuration:
>
> /props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,"0.1.2.3:9092");//
> //props.put(ConsumerConfig.GROUP_ID_CONFIG, "my_group");//
> //props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "true");//
> //props.put(ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG, "60000");//
> //props.put(ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG, "20000");//
> //props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG,
> "org.apache.kafka.common.serialization.IntegerDeserializer");//
> //props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,
> "org.apache.kafka.common.serialization.StringDeserializer");//
>
> /The broker configuration has no significant changes.
> I detect the error using Kafka from source code and tracing the methods
> manually, because no raise any exception when used from compiled .jars.
> What can i do in order to, at least, known more details about this error,
> or ideally solving it ;-)
>
> Thanks
>