You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Jay Kreps <ja...@gmail.com> on 2015/04/12 22:12:01 UTC

Exceptions on trunk

I see the following issues on trunk when doing almost anything:


1. Offset manager: it looks like this code has never been run, there is a
type that tries to read the offset as a string.

java.lang.ClassCastException: java.lang.Long cannot be cast to
java.lang.String

at
kafka.server.OffsetManager$.kafka$server$OffsetManager$$readMessageValue(OffsetManager.scala:576)

at
kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:388)

at
kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:377)

at scala.collection.Iterator$class.foreach(Iterator.scala:727)

at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32)

at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)

at kafka.message.MessageSet.foreach(MessageSet.scala:67)

at
kafka.server.OffsetManager.kafka$server$OffsetManager$$loadOffsets$1(OffsetManager.scala:377)

at
kafka.server.OffsetManager$$anonfun$loadOffsetsFromLog$1.apply$mcV$sp(OffsetManager.scala:358)

at
kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:108)

at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:60)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

at java.util.concurrent.FutureTask.run(FutureTask.java:262)

at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)

at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)

at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:744)



2. Errors in RequestChannel when updating metrics:

kafka.common.KafkaException: Wrong request type 12

at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)

at
kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)

at kafka.network.Processor.write(SocketServer.scala:514)

at kafka.network.Processor.run(SocketServer.scala:379)

at java.lang.Thread.run(Thread.java:744)

[2015-04-12 12:54:52,077] INFO [Kafka Coordinator 0]: Registered consumer
my-group-24 for group my-group (kafka.coordinator.ConsumerCoordinator)

[2015-04-12 12:54:52,080] INFO [Kafka Coordinator 0]: Handled join-group
from consumer  to group my-group (kafka.coordinator.ConsumerCoordinator)

[2015-04-12 12:54:52,081] ERROR Closing socket for /10.0.0.220 because of
error (kafka.network.Processor)

kafka.common.KafkaException: Wrong request type 11

at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)

at
kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)

at kafka.network.Processor.write(SocketServer.scala:514)

at kafka.network.Processor.run(SocketServer.scala:379)

 at java.lang.Thread.run(Thread.java:744)


-Jay

Re: Exceptions on trunk

Posted by Guozhang Wang <wa...@gmail.com>.
My bad on 1), fixing it right now.

On Sun, Apr 12, 2015 at 8:06 PM, Jun Rao <ju...@confluent.io> wrote:

> For 1, the issue seems to be that we try to get metadata using the offset
> field. Filed KAFKA-2117.
>
> Thanks,
>
> Jun
>
> On Sun, Apr 12, 2015 at 1:12 PM, Jay Kreps <ja...@gmail.com> wrote:
>
> > I see the following issues on trunk when doing almost anything:
> >
> >
> > 1. Offset manager: it looks like this code has never been run, there is a
> > type that tries to read the offset as a string.
> >
> > java.lang.ClassCastException: java.lang.Long cannot be cast to
> > java.lang.String
> >
> > at
> >
> >
> kafka.server.OffsetManager$.kafka$server$OffsetManager$$readMessageValue(OffsetManager.scala:576)
> >
> > at
> >
> >
> kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:388)
> >
> > at
> >
> >
> kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:377)
> >
> > at scala.collection.Iterator$class.foreach(Iterator.scala:727)
> >
> > at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32)
> >
> > at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> >
> > at kafka.message.MessageSet.foreach(MessageSet.scala:67)
> >
> > at
> >
> >
> kafka.server.OffsetManager.kafka$server$OffsetManager$$loadOffsets$1(OffsetManager.scala:377)
> >
> > at
> >
> >
> kafka.server.OffsetManager$$anonfun$loadOffsetsFromLog$1.apply$mcV$sp(OffsetManager.scala:358)
> >
> > at
> >
> >
> kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:108)
> >
> > at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:60)
> >
> > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >
> > at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> >
> > at
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> >
> > at
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> >
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >
> > at java.lang.Thread.run(Thread.java:744)
> >
> >
> >
> > 2. Errors in RequestChannel when updating metrics:
> >
> > kafka.common.KafkaException: Wrong request type 12
> >
> > at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
> >
> > at
> >
> >
> kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
> >
> > at kafka.network.Processor.write(SocketServer.scala:514)
> >
> > at kafka.network.Processor.run(SocketServer.scala:379)
> >
> > at java.lang.Thread.run(Thread.java:744)
> >
> > [2015-04-12 12:54:52,077] INFO [Kafka Coordinator 0]: Registered consumer
> > my-group-24 for group my-group (kafka.coordinator.ConsumerCoordinator)
> >
> > [2015-04-12 12:54:52,080] INFO [Kafka Coordinator 0]: Handled join-group
> > from consumer  to group my-group (kafka.coordinator.ConsumerCoordinator)
> >
> > [2015-04-12 12:54:52,081] ERROR Closing socket for /10.0.0.220 because
> of
> > error (kafka.network.Processor)
> >
> > kafka.common.KafkaException: Wrong request type 11
> >
> > at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
> >
> > at
> >
> >
> kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
> >
> > at kafka.network.Processor.write(SocketServer.scala:514)
> >
> > at kafka.network.Processor.run(SocketServer.scala:379)
> >
> >  at java.lang.Thread.run(Thread.java:744)
> >
> >
> > -Jay
> >
>



-- 
-- Guozhang

Re: Exceptions on trunk

Posted by Jun Rao <ju...@confluent.io>.
For 1, the issue seems to be that we try to get metadata using the offset
field. Filed KAFKA-2117.

Thanks,

Jun

On Sun, Apr 12, 2015 at 1:12 PM, Jay Kreps <ja...@gmail.com> wrote:

> I see the following issues on trunk when doing almost anything:
>
>
> 1. Offset manager: it looks like this code has never been run, there is a
> type that tries to read the offset as a string.
>
> java.lang.ClassCastException: java.lang.Long cannot be cast to
> java.lang.String
>
> at
>
> kafka.server.OffsetManager$.kafka$server$OffsetManager$$readMessageValue(OffsetManager.scala:576)
>
> at
>
> kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:388)
>
> at
>
> kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:377)
>
> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
>
> at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32)
>
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
>
> at kafka.message.MessageSet.foreach(MessageSet.scala:67)
>
> at
>
> kafka.server.OffsetManager.kafka$server$OffsetManager$$loadOffsets$1(OffsetManager.scala:377)
>
> at
>
> kafka.server.OffsetManager$$anonfun$loadOffsetsFromLog$1.apply$mcV$sp(OffsetManager.scala:358)
>
> at
>
> kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:108)
>
> at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:60)
>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:744)
>
>
>
> 2. Errors in RequestChannel when updating metrics:
>
> kafka.common.KafkaException: Wrong request type 12
>
> at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
>
> at
>
> kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
>
> at kafka.network.Processor.write(SocketServer.scala:514)
>
> at kafka.network.Processor.run(SocketServer.scala:379)
>
> at java.lang.Thread.run(Thread.java:744)
>
> [2015-04-12 12:54:52,077] INFO [Kafka Coordinator 0]: Registered consumer
> my-group-24 for group my-group (kafka.coordinator.ConsumerCoordinator)
>
> [2015-04-12 12:54:52,080] INFO [Kafka Coordinator 0]: Handled join-group
> from consumer  to group my-group (kafka.coordinator.ConsumerCoordinator)
>
> [2015-04-12 12:54:52,081] ERROR Closing socket for /10.0.0.220 because of
> error (kafka.network.Processor)
>
> kafka.common.KafkaException: Wrong request type 11
>
> at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
>
> at
>
> kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
>
> at kafka.network.Processor.write(SocketServer.scala:514)
>
> at kafka.network.Processor.run(SocketServer.scala:379)
>
>  at java.lang.Thread.run(Thread.java:744)
>
>
> -Jay
>

Re: Exceptions on trunk

Posted by Gwen Shapira <gs...@cloudera.com>.
The second error is a result of KAFKA-2044 - we moved few Requests out
of RequestKeys to the newer ApiKeys, but didn't update the metrics
code.

The only requests that moved were HeartBeat and JoinGroup, which are
part of the new consumer.
I think it will be a good idea to add tests that use new consumer to
system_test, so these issues will be caught by the automated
test-suit.

I'll open JIRAs for both the RequestKeys issue and the lack of system_tests.

Gwen

On Sun, Apr 12, 2015 at 1:12 PM, Jay Kreps <ja...@gmail.com> wrote:
> I see the following issues on trunk when doing almost anything:
>
>
> 1. Offset manager: it looks like this code has never been run, there is a
> type that tries to read the offset as a string.
>
> java.lang.ClassCastException: java.lang.Long cannot be cast to
> java.lang.String
>
> at
> kafka.server.OffsetManager$.kafka$server$OffsetManager$$readMessageValue(OffsetManager.scala:576)
>
> at
> kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:388)
>
> at
> kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:377)
>
> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
>
> at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32)
>
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
>
> at kafka.message.MessageSet.foreach(MessageSet.scala:67)
>
> at
> kafka.server.OffsetManager.kafka$server$OffsetManager$$loadOffsets$1(OffsetManager.scala:377)
>
> at
> kafka.server.OffsetManager$$anonfun$loadOffsetsFromLog$1.apply$mcV$sp(OffsetManager.scala:358)
>
> at
> kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:108)
>
> at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:60)
>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:744)
>
>
>
> 2. Errors in RequestChannel when updating metrics:
>
> kafka.common.KafkaException: Wrong request type 12
>
> at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
>
> at
> kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
>
> at kafka.network.Processor.write(SocketServer.scala:514)
>
> at kafka.network.Processor.run(SocketServer.scala:379)
>
> at java.lang.Thread.run(Thread.java:744)
>
> [2015-04-12 12:54:52,077] INFO [Kafka Coordinator 0]: Registered consumer
> my-group-24 for group my-group (kafka.coordinator.ConsumerCoordinator)
>
> [2015-04-12 12:54:52,080] INFO [Kafka Coordinator 0]: Handled join-group
> from consumer  to group my-group (kafka.coordinator.ConsumerCoordinator)
>
> [2015-04-12 12:54:52,081] ERROR Closing socket for /10.0.0.220 because of
> error (kafka.network.Processor)
>
> kafka.common.KafkaException: Wrong request type 11
>
> at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
>
> at
> kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
>
> at kafka.network.Processor.write(SocketServer.scala:514)
>
> at kafka.network.Processor.run(SocketServer.scala:379)
>
>  at java.lang.Thread.run(Thread.java:744)
>
>
> -Jay

Re: Exceptions on trunk

Posted by Jay Kreps <ja...@gmail.com>.
Also, I just want to remind people that everything in kafka.network should
really be a standalone network server, so that code block in RequestChannel
that hard codes stuff about followers is really wrong.

-Jay

On Sun, Apr 12, 2015 at 1:12 PM, Jay Kreps <ja...@gmail.com> wrote:

> I see the following issues on trunk when doing almost anything:
>
>
> 1. Offset manager: it looks like this code has never been run, there is a
> type that tries to read the offset as a string.
>
> java.lang.ClassCastException: java.lang.Long cannot be cast to
> java.lang.String
>
> at
> kafka.server.OffsetManager$.kafka$server$OffsetManager$$readMessageValue(OffsetManager.scala:576)
>
> at
> kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:388)
>
> at
> kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:377)
>
> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
>
> at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32)
>
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
>
> at kafka.message.MessageSet.foreach(MessageSet.scala:67)
>
> at
> kafka.server.OffsetManager.kafka$server$OffsetManager$$loadOffsets$1(OffsetManager.scala:377)
>
> at
> kafka.server.OffsetManager$$anonfun$loadOffsetsFromLog$1.apply$mcV$sp(OffsetManager.scala:358)
>
> at
> kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:108)
>
> at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:60)
>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:744)
>
>
>
> 2. Errors in RequestChannel when updating metrics:
>
> kafka.common.KafkaException: Wrong request type 12
>
> at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
>
> at
> kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
>
> at kafka.network.Processor.write(SocketServer.scala:514)
>
> at kafka.network.Processor.run(SocketServer.scala:379)
>
> at java.lang.Thread.run(Thread.java:744)
>
> [2015-04-12 12:54:52,077] INFO [Kafka Coordinator 0]: Registered consumer
> my-group-24 for group my-group (kafka.coordinator.ConsumerCoordinator)
>
> [2015-04-12 12:54:52,080] INFO [Kafka Coordinator 0]: Handled join-group
> from consumer  to group my-group (kafka.coordinator.ConsumerCoordinator)
>
> [2015-04-12 12:54:52,081] ERROR Closing socket for /10.0.0.220 because of
> error (kafka.network.Processor)
>
> kafka.common.KafkaException: Wrong request type 11
>
> at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
>
> at
> kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
>
> at kafka.network.Processor.write(SocketServer.scala:514)
>
> at kafka.network.Processor.run(SocketServer.scala:379)
>
>  at java.lang.Thread.run(Thread.java:744)
>
>
> -Jay
>