You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Jay Kreps <ja...@gmail.com> on 2015/04/12 22:12:01 UTC
Exceptions on trunk
I see the following issues on trunk when doing almost anything:
1. Offset manager: it looks like this code has never been run, there is a
type that tries to read the offset as a string.
java.lang.ClassCastException: java.lang.Long cannot be cast to
java.lang.String
at
kafka.server.OffsetManager$.kafka$server$OffsetManager$$readMessageValue(OffsetManager.scala:576)
at
kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:388)
at
kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:377)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at kafka.message.MessageSet.foreach(MessageSet.scala:67)
at
kafka.server.OffsetManager.kafka$server$OffsetManager$$loadOffsets$1(OffsetManager.scala:377)
at
kafka.server.OffsetManager$$anonfun$loadOffsetsFromLog$1.apply$mcV$sp(OffsetManager.scala:358)
at
kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:108)
at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:60)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
2. Errors in RequestChannel when updating metrics:
kafka.common.KafkaException: Wrong request type 12
at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
at
kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
at kafka.network.Processor.write(SocketServer.scala:514)
at kafka.network.Processor.run(SocketServer.scala:379)
at java.lang.Thread.run(Thread.java:744)
[2015-04-12 12:54:52,077] INFO [Kafka Coordinator 0]: Registered consumer
my-group-24 for group my-group (kafka.coordinator.ConsumerCoordinator)
[2015-04-12 12:54:52,080] INFO [Kafka Coordinator 0]: Handled join-group
from consumer to group my-group (kafka.coordinator.ConsumerCoordinator)
[2015-04-12 12:54:52,081] ERROR Closing socket for /10.0.0.220 because of
error (kafka.network.Processor)
kafka.common.KafkaException: Wrong request type 11
at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
at
kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
at kafka.network.Processor.write(SocketServer.scala:514)
at kafka.network.Processor.run(SocketServer.scala:379)
at java.lang.Thread.run(Thread.java:744)
-Jay
Re: Exceptions on trunk
Posted by Guozhang Wang <wa...@gmail.com>.
My bad on 1), fixing it right now.
On Sun, Apr 12, 2015 at 8:06 PM, Jun Rao <ju...@confluent.io> wrote:
> For 1, the issue seems to be that we try to get metadata using the offset
> field. Filed KAFKA-2117.
>
> Thanks,
>
> Jun
>
> On Sun, Apr 12, 2015 at 1:12 PM, Jay Kreps <ja...@gmail.com> wrote:
>
> > I see the following issues on trunk when doing almost anything:
> >
> >
> > 1. Offset manager: it looks like this code has never been run, there is a
> > type that tries to read the offset as a string.
> >
> > java.lang.ClassCastException: java.lang.Long cannot be cast to
> > java.lang.String
> >
> > at
> >
> >
> kafka.server.OffsetManager$.kafka$server$OffsetManager$$readMessageValue(OffsetManager.scala:576)
> >
> > at
> >
> >
> kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:388)
> >
> > at
> >
> >
> kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:377)
> >
> > at scala.collection.Iterator$class.foreach(Iterator.scala:727)
> >
> > at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32)
> >
> > at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> >
> > at kafka.message.MessageSet.foreach(MessageSet.scala:67)
> >
> > at
> >
> >
> kafka.server.OffsetManager.kafka$server$OffsetManager$$loadOffsets$1(OffsetManager.scala:377)
> >
> > at
> >
> >
> kafka.server.OffsetManager$$anonfun$loadOffsetsFromLog$1.apply$mcV$sp(OffsetManager.scala:358)
> >
> > at
> >
> >
> kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:108)
> >
> > at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:60)
> >
> > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >
> > at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> >
> > at
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> >
> > at
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> >
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >
> > at java.lang.Thread.run(Thread.java:744)
> >
> >
> >
> > 2. Errors in RequestChannel when updating metrics:
> >
> > kafka.common.KafkaException: Wrong request type 12
> >
> > at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
> >
> > at
> >
> >
> kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
> >
> > at kafka.network.Processor.write(SocketServer.scala:514)
> >
> > at kafka.network.Processor.run(SocketServer.scala:379)
> >
> > at java.lang.Thread.run(Thread.java:744)
> >
> > [2015-04-12 12:54:52,077] INFO [Kafka Coordinator 0]: Registered consumer
> > my-group-24 for group my-group (kafka.coordinator.ConsumerCoordinator)
> >
> > [2015-04-12 12:54:52,080] INFO [Kafka Coordinator 0]: Handled join-group
> > from consumer to group my-group (kafka.coordinator.ConsumerCoordinator)
> >
> > [2015-04-12 12:54:52,081] ERROR Closing socket for /10.0.0.220 because
> of
> > error (kafka.network.Processor)
> >
> > kafka.common.KafkaException: Wrong request type 11
> >
> > at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
> >
> > at
> >
> >
> kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
> >
> > at kafka.network.Processor.write(SocketServer.scala:514)
> >
> > at kafka.network.Processor.run(SocketServer.scala:379)
> >
> > at java.lang.Thread.run(Thread.java:744)
> >
> >
> > -Jay
> >
>
--
-- Guozhang
Re: Exceptions on trunk
Posted by Jun Rao <ju...@confluent.io>.
For 1, the issue seems to be that we try to get metadata using the offset
field. Filed KAFKA-2117.
Thanks,
Jun
On Sun, Apr 12, 2015 at 1:12 PM, Jay Kreps <ja...@gmail.com> wrote:
> I see the following issues on trunk when doing almost anything:
>
>
> 1. Offset manager: it looks like this code has never been run, there is a
> type that tries to read the offset as a string.
>
> java.lang.ClassCastException: java.lang.Long cannot be cast to
> java.lang.String
>
> at
>
> kafka.server.OffsetManager$.kafka$server$OffsetManager$$readMessageValue(OffsetManager.scala:576)
>
> at
>
> kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:388)
>
> at
>
> kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:377)
>
> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
>
> at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32)
>
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
>
> at kafka.message.MessageSet.foreach(MessageSet.scala:67)
>
> at
>
> kafka.server.OffsetManager.kafka$server$OffsetManager$$loadOffsets$1(OffsetManager.scala:377)
>
> at
>
> kafka.server.OffsetManager$$anonfun$loadOffsetsFromLog$1.apply$mcV$sp(OffsetManager.scala:358)
>
> at
>
> kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:108)
>
> at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:60)
>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:744)
>
>
>
> 2. Errors in RequestChannel when updating metrics:
>
> kafka.common.KafkaException: Wrong request type 12
>
> at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
>
> at
>
> kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
>
> at kafka.network.Processor.write(SocketServer.scala:514)
>
> at kafka.network.Processor.run(SocketServer.scala:379)
>
> at java.lang.Thread.run(Thread.java:744)
>
> [2015-04-12 12:54:52,077] INFO [Kafka Coordinator 0]: Registered consumer
> my-group-24 for group my-group (kafka.coordinator.ConsumerCoordinator)
>
> [2015-04-12 12:54:52,080] INFO [Kafka Coordinator 0]: Handled join-group
> from consumer to group my-group (kafka.coordinator.ConsumerCoordinator)
>
> [2015-04-12 12:54:52,081] ERROR Closing socket for /10.0.0.220 because of
> error (kafka.network.Processor)
>
> kafka.common.KafkaException: Wrong request type 11
>
> at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
>
> at
>
> kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
>
> at kafka.network.Processor.write(SocketServer.scala:514)
>
> at kafka.network.Processor.run(SocketServer.scala:379)
>
> at java.lang.Thread.run(Thread.java:744)
>
>
> -Jay
>
Re: Exceptions on trunk
Posted by Gwen Shapira <gs...@cloudera.com>.
The second error is a result of KAFKA-2044 - we moved few Requests out
of RequestKeys to the newer ApiKeys, but didn't update the metrics
code.
The only requests that moved were HeartBeat and JoinGroup, which are
part of the new consumer.
I think it will be a good idea to add tests that use new consumer to
system_test, so these issues will be caught by the automated
test-suit.
I'll open JIRAs for both the RequestKeys issue and the lack of system_tests.
Gwen
On Sun, Apr 12, 2015 at 1:12 PM, Jay Kreps <ja...@gmail.com> wrote:
> I see the following issues on trunk when doing almost anything:
>
>
> 1. Offset manager: it looks like this code has never been run, there is a
> type that tries to read the offset as a string.
>
> java.lang.ClassCastException: java.lang.Long cannot be cast to
> java.lang.String
>
> at
> kafka.server.OffsetManager$.kafka$server$OffsetManager$$readMessageValue(OffsetManager.scala:576)
>
> at
> kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:388)
>
> at
> kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:377)
>
> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
>
> at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32)
>
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
>
> at kafka.message.MessageSet.foreach(MessageSet.scala:67)
>
> at
> kafka.server.OffsetManager.kafka$server$OffsetManager$$loadOffsets$1(OffsetManager.scala:377)
>
> at
> kafka.server.OffsetManager$$anonfun$loadOffsetsFromLog$1.apply$mcV$sp(OffsetManager.scala:358)
>
> at
> kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:108)
>
> at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:60)
>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:744)
>
>
>
> 2. Errors in RequestChannel when updating metrics:
>
> kafka.common.KafkaException: Wrong request type 12
>
> at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
>
> at
> kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
>
> at kafka.network.Processor.write(SocketServer.scala:514)
>
> at kafka.network.Processor.run(SocketServer.scala:379)
>
> at java.lang.Thread.run(Thread.java:744)
>
> [2015-04-12 12:54:52,077] INFO [Kafka Coordinator 0]: Registered consumer
> my-group-24 for group my-group (kafka.coordinator.ConsumerCoordinator)
>
> [2015-04-12 12:54:52,080] INFO [Kafka Coordinator 0]: Handled join-group
> from consumer to group my-group (kafka.coordinator.ConsumerCoordinator)
>
> [2015-04-12 12:54:52,081] ERROR Closing socket for /10.0.0.220 because of
> error (kafka.network.Processor)
>
> kafka.common.KafkaException: Wrong request type 11
>
> at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
>
> at
> kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
>
> at kafka.network.Processor.write(SocketServer.scala:514)
>
> at kafka.network.Processor.run(SocketServer.scala:379)
>
> at java.lang.Thread.run(Thread.java:744)
>
>
> -Jay
Re: Exceptions on trunk
Posted by Jay Kreps <ja...@gmail.com>.
Also, I just want to remind people that everything in kafka.network should
really be a standalone network server, so that code block in RequestChannel
that hard codes stuff about followers is really wrong.
-Jay
On Sun, Apr 12, 2015 at 1:12 PM, Jay Kreps <ja...@gmail.com> wrote:
> I see the following issues on trunk when doing almost anything:
>
>
> 1. Offset manager: it looks like this code has never been run, there is a
> type that tries to read the offset as a string.
>
> java.lang.ClassCastException: java.lang.Long cannot be cast to
> java.lang.String
>
> at
> kafka.server.OffsetManager$.kafka$server$OffsetManager$$readMessageValue(OffsetManager.scala:576)
>
> at
> kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:388)
>
> at
> kafka.server.OffsetManager$$anonfun$kafka$server$OffsetManager$$loadOffsets$1$2.apply(OffsetManager.scala:377)
>
> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
>
> at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32)
>
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
>
> at kafka.message.MessageSet.foreach(MessageSet.scala:67)
>
> at
> kafka.server.OffsetManager.kafka$server$OffsetManager$$loadOffsets$1(OffsetManager.scala:377)
>
> at
> kafka.server.OffsetManager$$anonfun$loadOffsetsFromLog$1.apply$mcV$sp(OffsetManager.scala:358)
>
> at
> kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:108)
>
> at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:60)
>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:744)
>
>
>
> 2. Errors in RequestChannel when updating metrics:
>
> kafka.common.KafkaException: Wrong request type 12
>
> at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
>
> at
> kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
>
> at kafka.network.Processor.write(SocketServer.scala:514)
>
> at kafka.network.Processor.run(SocketServer.scala:379)
>
> at java.lang.Thread.run(Thread.java:744)
>
> [2015-04-12 12:54:52,077] INFO [Kafka Coordinator 0]: Registered consumer
> my-group-24 for group my-group (kafka.coordinator.ConsumerCoordinator)
>
> [2015-04-12 12:54:52,080] INFO [Kafka Coordinator 0]: Handled join-group
> from consumer to group my-group (kafka.coordinator.ConsumerCoordinator)
>
> [2015-04-12 12:54:52,081] ERROR Closing socket for /10.0.0.220 because of
> error (kafka.network.Processor)
>
> kafka.common.KafkaException: Wrong request type 11
>
> at kafka.api.RequestKeys$.nameForKey(RequestKeys.scala:55)
>
> at
> kafka.network.RequestChannel$Request.updateRequestMetrics(RequestChannel.scala:85)
>
> at kafka.network.Processor.write(SocketServer.scala:514)
>
> at kafka.network.Processor.run(SocketServer.scala:379)
>
> at java.lang.Thread.run(Thread.java:744)
>
>
> -Jay
>