You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by anand nalya <an...@gmail.com> on 2013/06/24 06:30:47 UTC

Error when restarting consumer

Hi,

I'm using High Level Consumer with kafka 0.7.2. The consumer is able to
consume messages from kafka for the first time but when I restart the
consumer process, I'm getting the following error:

2013-06-24 09:48:25 FetchRunnable-0 [ERROR] FetcherRunnable:89 - error in
FetcherRunnable
kafka.network.InvalidRequestException: -2147400679 is not a valid request
size.
    at
kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:60)
    at kafka.network.Receive$class.readCompletely(Transmission.scala:55)
    at
kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
    at kafka.consumer.SimpleConsumer.getResponse(SimpleConsumer.scala:177)
    at kafka.consumer.SimpleConsumer.liftedTree2$1(SimpleConsumer.scala:117)
    at kafka.consumer.SimpleConsumer.multifetch(SimpleConsumer.scala:115)
    at kafka.consumer.FetcherRunnable.run(FetcherRunnable.scala:60)

I'm not sure how is it getting -2147400679 as the request size. Any ideas
or pointers what may be wrong here?

Thanks,
Anand

Re: Error when restarting consumer

Posted by Jun Rao <ju...@gmail.com>.
Are you consuming more than 2GB of data for a single fetch request (for all
partitions)? If so, this can cause overflow in the request (response) size
since it's represented as an int. The solution is to reduce the fetch size.

Thanks,

Jun


On Sun, Jun 23, 2013 at 9:30 PM, anand nalya <an...@gmail.com> wrote:

> Hi,
>
> I'm using High Level Consumer with kafka 0.7.2. The consumer is able to
> consume messages from kafka for the first time but when I restart the
> consumer process, I'm getting the following error:
>
> 2013-06-24 09:48:25 FetchRunnable-0 [ERROR] FetcherRunnable:89 - error in
> FetcherRunnable
> kafka.network.InvalidRequestException: -2147400679 is not a valid request
> size.
>     at
>
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:60)
>     at kafka.network.Receive$class.readCompletely(Transmission.scala:55)
>     at
>
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
>     at kafka.consumer.SimpleConsumer.getResponse(SimpleConsumer.scala:177)
>     at
> kafka.consumer.SimpleConsumer.liftedTree2$1(SimpleConsumer.scala:117)
>     at kafka.consumer.SimpleConsumer.multifetch(SimpleConsumer.scala:115)
>     at kafka.consumer.FetcherRunnable.run(FetcherRunnable.scala:60)
>
> I'm not sure how is it getting -2147400679 as the request size. Any ideas
> or pointers what may be wrong here?
>
> Thanks,
> Anand
>