You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by 揣立武 <ch...@meituan.com> on 2017/02/16 06:29:25 UTC
Kafka exception
Hi,all! Our program uses the high level consumer api(the version is 0.8.x). Sometimes the program will throw an exception in the 42th row in kafka.utils.IteratorTemplate class,the content is "throw new IllegalStateException("Expected item but none found.")".
I think it is a race condition problem between the close thread and the consume thread. When the close thread calling the method ConsumerConnector.shutdown(), it will set ConsumerIterator's state is NOT_READY. But at the same time, the consume thread calls the method ConsumerIterator.hasNext() and goes to the 67th row in kafka.utils.IteratorTemplate class,the content is "if(state == DONE) {", the if will be false that means has a item. And when calling the ConsumerIterator.next(), it will throw that exception.
Have you ever had this problem? Please tell me how to deal with it, thanks!
Re: Kafka exception
Posted by Yuanjia <yu...@163.com>.
Hi Liwu,
Correct me if I am wrong.
When calling the method ConsumerConnector.shutdown(), it will send "ZookeeperConsumerConnector.shutdownCommand" to the queue, not set ConsumerIterator's state is NOT_READY directly. So the consumer will continue consuming until get the shutdownCommand in the queue.
Is there any exception information when calling the method ConsumerIterator.makeNext()?
Yuanjia Li
From: 揣立武
Date: 2017-02-16 14:29
To: dev
CC: users; 陈希; 涂扬
Subject: Kafka exception
Hi,all! Our program uses the high level consumer api(the version is 0.8.x). Sometimes the program will throw an exception in the 42th row in kafka.utils.IteratorTemplate class,the content is "throw new IllegalStateException("Expected item but none found.")".
I think it is a race condition problem between the close thread and the consume thread. When the close thread calling the method ConsumerConnector.shutdown(), it will set ConsumerIterator's state is NOT_READY. But at the same time, the consume thread calls the method ConsumerIterator.hasNext() and goes to the 67th row in kafka.utils.IteratorTemplate class,the content is "if(state == DONE) {", the if will be false that means has a item. And when calling the ConsumerIterator.next(), it will throw that exception.
Have you ever had this problem? Please tell me how to deal with it, thanks!
Re: Kafka exception
Posted by Yuanjia <yu...@163.com>.
Hi Liwu,
Correct me if I am wrong.
When calling the method ConsumerConnector.shutdown(), it will send "ZookeeperConsumerConnector.shutdownCommand" to the queue, not set ConsumerIterator's state is NOT_READY directly. So the consumer will continue consuming until get the shutdownCommand in the queue.
Is there any exception information when calling the method ConsumerIterator.makeNext()?
Yuanjia Li
From: 揣立武
Date: 2017-02-16 14:29
To: dev
CC: users; 陈希; 涂扬
Subject: Kafka exception
Hi,all! Our program uses the high level consumer api(the version is 0.8.x). Sometimes the program will throw an exception in the 42th row in kafka.utils.IteratorTemplate class,the content is "throw new IllegalStateException("Expected item but none found.")".
I think it is a race condition problem between the close thread and the consume thread. When the close thread calling the method ConsumerConnector.shutdown(), it will set ConsumerIterator's state is NOT_READY. But at the same time, the consume thread calls the method ConsumerIterator.hasNext() and goes to the 67th row in kafka.utils.IteratorTemplate class,the content is "if(state == DONE) {", the if will be false that means has a item. And when calling the ConsumerIterator.next(), it will throw that exception.
Have you ever had this problem? Please tell me how to deal with it, thanks!
Re: Kafka exception
Posted by Colin McCabe <cm...@apache.org>.
Can you reproduce this with the latest version of Kafka?
best,
Colin
On Wed, Feb 15, 2017, at 22:29, 揣立武 wrote:
>
> Hi,all! Our program uses the high level consumer api(the version is
> 0.8.x). Sometimes the program will throw an exception in the 42th row in
> kafka.utils.IteratorTemplate class,the content is "throw new
> IllegalStateException("Expected item but none found.")".
>
> I think it is a race condition problem between the close thread and the
> consume thread. When the close thread calling the method
> ConsumerConnector.shutdown(), it will set ConsumerIterator's state is
> NOT_READY. But at the same time, the consume thread calls the method
> ConsumerIterator.hasNext() and goes to the 67th row in
> kafka.utils.IteratorTemplate class,the content is "if(state == DONE) {",
> the if will be false that means has a item. And when calling the
> ConsumerIterator.next(), it will throw that exception.
>
> Have you ever had this problem? Please tell me how to deal with it,
> thanks!
>
>
>