You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Bill Bejeck <bb...@gmail.com> on 2015/11/27 15:19:31 UTC
Possible StreamingConfig Bug
All,
When starting KafkaStreaming I'm getting the following error (even when
explicitly setting the groupId with props.put("group.id","test-consumer-group")
);
Exception in thread "StreamThread-1"
org.apache.kafka.common.KafkaException:
org.apache.kafka.common.errors.ApiException: The configured groupId is
invalid
at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:309)
at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:198)
Caused by: org.apache.kafka.common.errors.ApiException: The configured
groupId is invalid
I've traced the source of the issue to the
StreamingConfig.getConsumerConfigs method as it calls
getRestoreConsumerConfigs (which explicitly removes the groupId property)
vs using getBaseConsumerConfigs which returns the passed in configs
unaltered.
When I switched the method call, KafkaStreaming starts up fine.
If you agree with this change/fix, I'll create a Jira ticket and put in the
PR, yada yada yada......
Thanks,
Bill
Re: Possible StreamingConfig Bug
Posted by Bill Bejeck <bb...@gmail.com>.
Yes, that is the case from my usage.
FWIW I have a Jira ticket (KAFKA-2902) and have submitted a PR (#596)
to address the issue.
Thanks,
Bill
> On Nov 28, 2015, at 1:27 AM, Guozhang Wang <wa...@gmail.com> wrote:
>
> I think the problem is that "getConsumerConfigs" called in StreamThread.
> createConsumer() triggers getRestoreConsumerConfigs instead of
> getBaseConsumerConfigs, which seems a bug to me.
>
> Guozhang
>
> On Fri, Nov 27, 2015 at 8:01 PM, Yasuhiro Matsuda <
> yasuhiro.matsuda@gmail.com> wrote:
>
>> The group id is removed from the restore consumer config because the
>> restore consumer should not participate in the specified consumer group. I
>> don't know why it is failing.
>>
>> On Fri, Nov 27, 2015 at 12:37 PM, Guozhang Wang <wa...@gmail.com>
>> wrote:
>>
>>> Hello Bill,
>>>
>>> Thanks for reporting it, this is a valid issue, could you create a
>> ticket?
>>>
>>> Guozhang
>>>
>>>> On Fri, Nov 27, 2015 at 6:19 AM, Bill Bejeck <bb...@gmail.com> wrote:
>>>>
>>>> All,
>>>>
>>>> When starting KafkaStreaming I'm getting the following error (even when
>>>> explicitly setting the groupId with props.put("group.id
>>>> ","test-consumer-group")
>>>> );
>>>>
>>>> Exception in thread "StreamThread-1"
>>>> org.apache.kafka.common.KafkaException:
>>>> org.apache.kafka.common.errors.ApiException: The configured groupId is
>>>> invalid
>>>> at
>> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:309)
>>>> at
>> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:198)
>>>> Caused by: org.apache.kafka.common.errors.ApiException: The configured
>>>> groupId is invalid
>>>>
>>>> I've traced the source of the issue to the
>>>> StreamingConfig.getConsumerConfigs method as it calls
>>>> getRestoreConsumerConfigs (which explicitly removes the groupId
>> property)
>>>> vs using getBaseConsumerConfigs which returns the passed in configs
>>>> unaltered.
>>>>
>>>> When I switched the method call, KafkaStreaming starts up fine.
>>>>
>>>> If you agree with this change/fix, I'll create a Jira ticket and put in
>>> the
>>>> PR, yada yada yada......
>>>>
>>>> Thanks,
>>>> Bill
>>>
>>>
>>>
>>> --
>>> -- Guozhang
>
>
>
> --
> -- Guozhang
Re: Possible StreamingConfig Bug
Posted by Guozhang Wang <wa...@gmail.com>.
I think the problem is that "getConsumerConfigs" called in StreamThread.
createConsumer() triggers getRestoreConsumerConfigs instead of
getBaseConsumerConfigs, which seems a bug to me.
Guozhang
On Fri, Nov 27, 2015 at 8:01 PM, Yasuhiro Matsuda <
yasuhiro.matsuda@gmail.com> wrote:
> The group id is removed from the restore consumer config because the
> restore consumer should not participate in the specified consumer group. I
> don't know why it is failing.
>
> On Fri, Nov 27, 2015 at 12:37 PM, Guozhang Wang <wa...@gmail.com>
> wrote:
>
> > Hello Bill,
> >
> > Thanks for reporting it, this is a valid issue, could you create a
> ticket?
> >
> > Guozhang
> >
> > On Fri, Nov 27, 2015 at 6:19 AM, Bill Bejeck <bb...@gmail.com> wrote:
> >
> > > All,
> > >
> > > When starting KafkaStreaming I'm getting the following error (even when
> > > explicitly setting the groupId with props.put("group.id
> > > ","test-consumer-group")
> > > );
> > >
> > > Exception in thread "StreamThread-1"
> > > org.apache.kafka.common.KafkaException:
> > > org.apache.kafka.common.errors.ApiException: The configured groupId is
> > > invalid
> > > at
> > >
> > >
> >
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:309)
> > > at
> > >
> > >
> >
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:198)
> > > Caused by: org.apache.kafka.common.errors.ApiException: The configured
> > > groupId is invalid
> > >
> > > I've traced the source of the issue to the
> > > StreamingConfig.getConsumerConfigs method as it calls
> > > getRestoreConsumerConfigs (which explicitly removes the groupId
> property)
> > > vs using getBaseConsumerConfigs which returns the passed in configs
> > > unaltered.
> > >
> > > When I switched the method call, KafkaStreaming starts up fine.
> > >
> > > If you agree with this change/fix, I'll create a Jira ticket and put in
> > the
> > > PR, yada yada yada......
> > >
> > > Thanks,
> > > Bill
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
--
-- Guozhang
Re: Possible StreamingConfig Bug
Posted by Yasuhiro Matsuda <ya...@gmail.com>.
The group id is removed from the restore consumer config because the
restore consumer should not participate in the specified consumer group. I
don't know why it is failing.
On Fri, Nov 27, 2015 at 12:37 PM, Guozhang Wang <wa...@gmail.com> wrote:
> Hello Bill,
>
> Thanks for reporting it, this is a valid issue, could you create a ticket?
>
> Guozhang
>
> On Fri, Nov 27, 2015 at 6:19 AM, Bill Bejeck <bb...@gmail.com> wrote:
>
> > All,
> >
> > When starting KafkaStreaming I'm getting the following error (even when
> > explicitly setting the groupId with props.put("group.id
> > ","test-consumer-group")
> > );
> >
> > Exception in thread "StreamThread-1"
> > org.apache.kafka.common.KafkaException:
> > org.apache.kafka.common.errors.ApiException: The configured groupId is
> > invalid
> > at
> >
> >
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:309)
> > at
> >
> >
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:198)
> > Caused by: org.apache.kafka.common.errors.ApiException: The configured
> > groupId is invalid
> >
> > I've traced the source of the issue to the
> > StreamingConfig.getConsumerConfigs method as it calls
> > getRestoreConsumerConfigs (which explicitly removes the groupId property)
> > vs using getBaseConsumerConfigs which returns the passed in configs
> > unaltered.
> >
> > When I switched the method call, KafkaStreaming starts up fine.
> >
> > If you agree with this change/fix, I'll create a Jira ticket and put in
> the
> > PR, yada yada yada......
> >
> > Thanks,
> > Bill
> >
>
>
>
> --
> -- Guozhang
>
Re: Possible StreamingConfig Bug
Posted by Guozhang Wang <wa...@gmail.com>.
Hello Bill,
Thanks for reporting it, this is a valid issue, could you create a ticket?
Guozhang
On Fri, Nov 27, 2015 at 6:19 AM, Bill Bejeck <bb...@gmail.com> wrote:
> All,
>
> When starting KafkaStreaming I'm getting the following error (even when
> explicitly setting the groupId with props.put("group.id
> ","test-consumer-group")
> );
>
> Exception in thread "StreamThread-1"
> org.apache.kafka.common.KafkaException:
> org.apache.kafka.common.errors.ApiException: The configured groupId is
> invalid
> at
>
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:309)
> at
>
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:198)
> Caused by: org.apache.kafka.common.errors.ApiException: The configured
> groupId is invalid
>
> I've traced the source of the issue to the
> StreamingConfig.getConsumerConfigs method as it calls
> getRestoreConsumerConfigs (which explicitly removes the groupId property)
> vs using getBaseConsumerConfigs which returns the passed in configs
> unaltered.
>
> When I switched the method call, KafkaStreaming starts up fine.
>
> If you agree with this change/fix, I'll create a Jira ticket and put in the
> PR, yada yada yada......
>
> Thanks,
> Bill
>
--
-- Guozhang