You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Frank Lyaruu <fl...@gmail.com> on 2016/12/16 11:35:28 UTC
Error in Kafka Streams (that shouldn't be there)
Hi people,
I'm running a Kafka Streams app (trunk build from a few days ago), I see
this error from time to time:
Exception in thread "StreamThread-18" java.lang.NullPointerException
at org.apache.kafka.streams.processor.internals.StreamThread$5.
apply(StreamThread.java:998)
at org.apache.kafka.streams.processor.internals.StreamThread.pe
rformOnAllTasks(StreamThread.java:489)
at org.apache.kafka.streams.processor.internals.StreamThread.
closeAllTasks(StreamThread.java:995)
at org.apache.kafka.streams.processor.internals.StreamThread.sh
utdownTasksAndState(StreamThread.java:442)
at org.apache.kafka.streams.processor.internals.StreamThread.
shutdown(StreamThread.java:387)
at org.apache.kafka.streams.processor.internals.StreamThread.
run(StreamThread.java:368)
After that it spews a bunch of these:
Exception in thread "StreamThread-13" java.lang.IllegalStateException:
Incorrect state transition from RUNNING to NOT_RUNNING
at org.apache.kafka.streams.processor.internals.StreamThread.
setState(StreamThread.java:163)
at org.apache.kafka.streams.processor.internals.StreamThread.
shutdown(StreamThread.java:416)
I'm now running on multiple instances, before I ran on a single instance, I
don't think I saw it then.
regards, Frank
Re: Error in Kafka Streams (that shouldn't be there)
Posted by Frank Lyaruu <fl...@gmail.com>.
Hi Eno,
Just when I'm running.
I think the root cause is losing a connection to zookeeper, I occasionally
see: "Terminate ZkClient event thread."
In the zookeeper logs I see something like:
2016-12-21 11:40:32,915 [myid:] - WARN [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid
0x158f872be151aa6, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
at java.lang.Thread.run(Thread.java:745)
So if this is indeed the case, I'll probably need to move my zookeeper
instance to a quieter machine (?), but the problem remains: Why is a
dropped connection so lethal? My Kafka streams instance never seems to
recover.
On Fri, Dec 16, 2016 at 6:25 PM, Eno Thereska <en...@gmail.com>
wrote:
> Hi Frank,
>
> Is this happening when you are shutting down the app, or is the app
> shutting down unexpectedly?
>
> The state transition error is not the main issue, it's largely a side
> effect of the unexpected transition so the first error is the cause
> probably. However, I can't tell what happened before since a normal
> shutdown also looks the same.
>
> Eno
> > On 16 Dec 2016, at 11:35, Frank Lyaruu <fl...@gmail.com> wrote:
> >
> > Hi people,
> >
> > I'm running a Kafka Streams app (trunk build from a few days ago), I see
> > this error from time to time:
> >
> > Exception in thread "StreamThread-18" java.lang.NullPointerException
> >
> > at org.apache.kafka.streams.processor.internals.StreamThread$5.
> > apply(StreamThread.java:998)
> > at org.apache.kafka.streams.processor.internals.StreamThread.pe
> > rformOnAllTasks(StreamThread.java:489)
> > at org.apache.kafka.streams.processor.internals.StreamThread.
> > closeAllTasks(StreamThread.java:995)
> > at org.apache.kafka.streams.processor.internals.StreamThread.sh
> > utdownTasksAndState(StreamThread.java:442)
> > at org.apache.kafka.streams.processor.internals.StreamThread.
> > shutdown(StreamThread.java:387)
> > at org.apache.kafka.streams.processor.internals.StreamThread.
> > run(StreamThread.java:368)
> >
> >
> > After that it spews a bunch of these:
> >
> > Exception in thread "StreamThread-13" java.lang.IllegalStateException:
> > Incorrect state transition from RUNNING to NOT_RUNNING
> >
> > at org.apache.kafka.streams.processor.internals.StreamThread.
> > setState(StreamThread.java:163)
> > at org.apache.kafka.streams.processor.internals.StreamThread.
> > shutdown(StreamThread.java:416)
> >
> > I'm now running on multiple instances, before I ran on a single
> instance, I
> > don't think I saw it then.
> >
> > regards, Frank
>
>
Re: Error in Kafka Streams (that shouldn't be there)
Posted by Eno Thereska <en...@gmail.com>.
Hi Frank,
Is this happening when you are shutting down the app, or is the app shutting down unexpectedly?
The state transition error is not the main issue, it's largely a side effect of the unexpected transition so the first error is the cause probably. However, I can't tell what happened before since a normal shutdown also looks the same.
Eno
> On 16 Dec 2016, at 11:35, Frank Lyaruu <fl...@gmail.com> wrote:
>
> Hi people,
>
> I'm running a Kafka Streams app (trunk build from a few days ago), I see
> this error from time to time:
>
> Exception in thread "StreamThread-18" java.lang.NullPointerException
>
> at org.apache.kafka.streams.processor.internals.StreamThread$5.
> apply(StreamThread.java:998)
> at org.apache.kafka.streams.processor.internals.StreamThread.pe
> rformOnAllTasks(StreamThread.java:489)
> at org.apache.kafka.streams.processor.internals.StreamThread.
> closeAllTasks(StreamThread.java:995)
> at org.apache.kafka.streams.processor.internals.StreamThread.sh
> utdownTasksAndState(StreamThread.java:442)
> at org.apache.kafka.streams.processor.internals.StreamThread.
> shutdown(StreamThread.java:387)
> at org.apache.kafka.streams.processor.internals.StreamThread.
> run(StreamThread.java:368)
>
>
> After that it spews a bunch of these:
>
> Exception in thread "StreamThread-13" java.lang.IllegalStateException:
> Incorrect state transition from RUNNING to NOT_RUNNING
>
> at org.apache.kafka.streams.processor.internals.StreamThread.
> setState(StreamThread.java:163)
> at org.apache.kafka.streams.processor.internals.StreamThread.
> shutdown(StreamThread.java:416)
>
> I'm now running on multiple instances, before I ran on a single instance, I
> don't think I saw it then.
>
> regards, Frank