You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Meghana Narasimhan <mn...@bandwidth.com> on 2016/05/16 21:12:50 UTC

Mirrormaker schema exception

Hi,
I came across the following mirrormaker issue today which caused it to
shutdown. Don't see any indications of any issues in any of the other logs.
Any insight on this error will be much appreciated. Kafka version is
0.9.0.1 (confluent platform 2.0.1).

FATAL [mirrormaker-thread-0] Mirror maker thread failure due to
 (kafka.tools.MirrorMaker$MirrorMakerThread)
org.apache.kafka.common.protocol.types.SchemaException: Error reading field
'responses': Error reading array of size 488033, only 76 bytes available
        at
org.apache.kafka.common.protocol.types.Schema.read(Schema.java:71)
        at
org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:439)
        at
org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:265)
        at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:320)
        at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:213)
        at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:193)
        at
org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:908)
        at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:853)
        at
kafka.tools.MirrorMaker$MirrorMakerNewConsumer.receive(MirrorMaker.scala:526)
        at
kafka.tools.MirrorMaker$MirrorMakerThread.run(MirrorMaker.scala:395)

Thanks,
Meghana

Re: Mirrormaker schema exception

Posted by Meghana Narasimhan <mn...@bandwidth.com>.
Thanks for the reply Gwen. I have verified that the source and destination
clusters are both running the same version , 0.9.0.1-cp1. So the brokers
and MirrorMaker are all on the same version. Can the mirrormaker not be of
the same version ?  MirrorMaker has been running fine since we upgraded to
this version beginning of March, so not sure what caused it to fail on this
error yesterday.

Thanks,
Meghana

On Mon, May 16, 2016 at 9:10 PM, Gwen Shapira <gw...@confluent.io> wrote:

> It looks like you are using MirrorMaker from 0.9.0.1 while the source
> broker is older.
>
> MirrorMaker needs to be older than the older broker involved in the
> replication.
>
> Gwen
>
> On Mon, May 16, 2016 at 2:12 PM, Meghana Narasimhan
> <mn...@bandwidth.com> wrote:
> > Hi,
> > I came across the following mirrormaker issue today which caused it to
> > shutdown. Don't see any indications of any issues in any of the other
> logs.
> > Any insight on this error will be much appreciated. Kafka version is
> > 0.9.0.1 (confluent platform 2.0.1).
> >
> > FATAL [mirrormaker-thread-0] Mirror maker thread failure due to
> >  (kafka.tools.MirrorMaker$MirrorMakerThread)
> > org.apache.kafka.common.protocol.types.SchemaException: Error reading
> field
> > 'responses': Error reading array of size 488033, only 76 bytes available
> >         at
> > org.apache.kafka.common.protocol.types.Schema.read(Schema.java:71)
> >         at
> >
> org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:439)
> >         at
> > org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:265)
> >         at
> >
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:320)
> >         at
> >
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:213)
> >         at
> >
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:193)
> >         at
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:908)
> >         at
> >
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:853)
> >         at
> >
> kafka.tools.MirrorMaker$MirrorMakerNewConsumer.receive(MirrorMaker.scala:526)
> >         at
> > kafka.tools.MirrorMaker$MirrorMakerThread.run(MirrorMaker.scala:395)
> >
> > Thanks,
> > Meghana
>

Re: Mirrormaker schema exception

Posted by Gwen Shapira <gw...@confluent.io>.
It looks like you are using MirrorMaker from 0.9.0.1 while the source
broker is older.

MirrorMaker needs to be older than the older broker involved in the replication.

Gwen

On Mon, May 16, 2016 at 2:12 PM, Meghana Narasimhan
<mn...@bandwidth.com> wrote:
> Hi,
> I came across the following mirrormaker issue today which caused it to
> shutdown. Don't see any indications of any issues in any of the other logs.
> Any insight on this error will be much appreciated. Kafka version is
> 0.9.0.1 (confluent platform 2.0.1).
>
> FATAL [mirrormaker-thread-0] Mirror maker thread failure due to
>  (kafka.tools.MirrorMaker$MirrorMakerThread)
> org.apache.kafka.common.protocol.types.SchemaException: Error reading field
> 'responses': Error reading array of size 488033, only 76 bytes available
>         at
> org.apache.kafka.common.protocol.types.Schema.read(Schema.java:71)
>         at
> org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:439)
>         at
> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:265)
>         at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:320)
>         at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:213)
>         at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:193)
>         at
> org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:908)
>         at
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:853)
>         at
> kafka.tools.MirrorMaker$MirrorMakerNewConsumer.receive(MirrorMaker.scala:526)
>         at
> kafka.tools.MirrorMaker$MirrorMakerThread.run(MirrorMaker.scala:395)
>
> Thanks,
> Meghana