You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by David Ross <dy...@klout.com> on 2012/01/05 02:33:47 UTC
Re: [0.6] Consumer API Question - What if a ConsumerIterator has an error?
Another problem - just taking toString of an iterator in a failed state
throws an exception.
On Thu, Dec 22, 2011 at 1:40 AM, David Ross <dy...@klout.com> wrote:
> Howdy,
>
> I have come across a potential issue with the Consumer API in Kafka 0.6.
> After some tests and reading through the code, I believe the following
> scenario could result in an unhappy situation:
>
> 1. Create 5 streams listening to a topic
> 2. One of the streams times out (or has some other error) and it's
> iterator is now in mode FAILED
> 3. (I think) Messages will now come into all five streams, but one of
> them won't yield any new messages because of (2)
>
> The only solution I can think of is if an error is detected in any of the
> streams, all of them should be stopped and a new set of streams should be
> made. This seems unfavorable and would require messy concurrent programming.
>
> So is this an actual problem? Is there a better workaround? Have I
> misinterpreted something?
>
>
> Thanks,
>
> David
>
Re: [0.6] Consumer API Question - What if a ConsumerIterator has an error?
Posted by Neha Narkhede <ne...@gmail.com>.
Hi David,
Does this problem exist with the latest stable release of Kafka ?
http://incubator.apache.org/kafka/downloads.html
Thanks,
Neha
On Wed, Jan 4, 2012 at 5:33 PM, David Ross <dy...@klout.com> wrote:
> Another problem - just taking toString of an iterator in a failed state
> throws an exception.
>
> On Thu, Dec 22, 2011 at 1:40 AM, David Ross <dy...@klout.com> wrote:
>
>> Howdy,
>>
>> I have come across a potential issue with the Consumer API in Kafka 0.6.
>> After some tests and reading through the code, I believe the following
>> scenario could result in an unhappy situation:
>>
>> 1. Create 5 streams listening to a topic
>> 2. One of the streams times out (or has some other error) and it's
>> iterator is now in mode FAILED
>> 3. (I think) Messages will now come into all five streams, but one of
>> them won't yield any new messages because of (2)
>>
>> The only solution I can think of is if an error is detected in any of the
>> streams, all of them should be stopped and a new set of streams should be
>> made. This seems unfavorable and would require messy concurrent programming.
>>
>> So is this an actual problem? Is there a better workaround? Have I
>> misinterpreted something?
>>
>>
>> Thanks,
>>
>> David
>>