You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by srdo <gi...@git.apache.org> on 2018/05/11 17:53:49 UTC
[GitHub] storm pull request #2673: STORM-3070: Rewind buffer position if MessageDecod...
GitHub user srdo opened a pull request:
https://github.com/apache/storm/pull/2673
STORM-3070: Rewind buffer position if MessageDecoder encounters a Bac…
…kpressureStatus message code, but the rest of the message hasn't been received yet
https://issues.apache.org/jira/browse/STORM-3070
I haven't actually encountered a bug here, just happened across this line. I'm reasonably sure we need to rewind the buffer before returning.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/srdo/storm master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/2673.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2673
----
commit eae4dc216fc1cc858be929049d27254ad6ec0b8f
Author: Stig Rohde Døssing <sr...@...>
Date: 2018-05-11T17:50:48Z
STORM-3070: Rewind buffer position if MessageDecoder encounters a BackpressureStatus message code, but the rest of the message hasn't been received yet
----
---
[GitHub] storm pull request #2673: STORM-3070: Rewind buffer position if MessageDecod...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/storm/pull/2673
---
[GitHub] storm issue #2673: STORM-3070: Rewind buffer position if MessageDecoder enco...
Posted by srdo <gi...@git.apache.org>.
Github user srdo commented on the issue:
https://github.com/apache/storm/pull/2673
Okay, pretty sure I've seen this break now. Ran the TVL topology and got occasional worker crashes, e.g.
```
2018-05-15 16:36:01.018 o.a.s.m.n.StormClientHandler client-worker-1 [INFO] Connection to DESKTOP-AGC8TKM/10.0.75.1:6700 failed:
io.netty.handler.codec.DecoderException: java.lang.ClassCastException: java.util.ArrayList cannot be cast to org.apache.storm.messaging.netty.BackPressureStatus
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459) ~[netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
Caused by: java.lang.ClassCastException: java.util.ArrayList cannot be cast to org.apache.storm.messaging.netty.BackPressureStatus
at org.apache.storm.messaging.netty.BackPressureStatus.read(BackPressureStatus.java:56) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.messaging.netty.MessageDecoder.decode(MessageDecoder.java:121) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[netty-all-4.1.24.Final.jar:4.1.24.Final]
... 15 more
2018-05-15 16:36:01.040 o.a.s.m.n.StormClientHandler client-worker-1 [INFO] Connection to DESKTOP-AGC8TKM/10.0.75.1:6700 failed:
com.esotericsoftware.kryo.KryoException: Unable to find class: :
at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:156) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:781) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:689) ~[kryo-3.0.3.jar:?]
at org.apache.storm.serialization.KryoValuesDeserializer.deserializeFrom(KryoValuesDeserializer.java:31) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.serialization.KryoValuesDeserializer.deserialize(KryoValuesDeserializer.java:37) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.messaging.netty.StormClientHandler.channelRead(StormClientHandler.java:69) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884) [netty-all-4.1.24.Final.jar:4.1.24.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
Caused by: java.lang.ClassNotFoundException: :
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_144]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_144]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) ~[?:1.8.0_144]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_144]
at java.lang.Class.forName0(Native Method) ~[?:1.8.0_144]
at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_144]
at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:154) ~[kryo-3.0.3.jar:?]
... 28 more
```
Applying this change and rerunning seems to have fixed it.
---