You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by hiloboy0119 <gi...@git.apache.org> on 2014/03/03 21:09:33 UTC
[GitHub] incubator-storm pull request: Fixing an integer overflow in getSle...
GitHub user hiloboy0119 opened a pull request:
https://github.com/apache/incubator-storm/pull/41
Fixing an integer overflow in getSleepTimeMs().
This should fix the issue of negative values passed to sleep, causing IllegalArgumentExceptions.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hiloboy0119/incubator-storm master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-storm/pull/41.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #41
----
commit d9d637eab5f3761a4768fc960b2574d45962b466
Author: Drew <dr...@gradientx.com>
Date: 2014-03-03T20:08:43Z
Fixing an integer overflow in getSleepTimeMs(). This should fix the
issue of negative values passed to sleep, causing
IllegalArgumentExceptions.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: Fixing an integer overflow in getSle...
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on the pull request:
https://github.com/apache/incubator-storm/pull/41#issuecomment-36756596
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: Fixing an integer overflow in getSle...
Posted by neeleshs <gi...@git.apache.org>.
Github user neeleshs commented on the pull request:
https://github.com/apache/incubator-storm/pull/41#issuecomment-45402968
I have a topology with kafkaspout (parallelism 3, 3 partitions in the topic). Im using storm 0.9.1 tag and have built it with this patch. That got rid of the negative timeout error. But every time a message is sent from , one of the workers dies with this message:
```
2014-06-07 06:43:27 b.s.d.task [INFO] Emitting: processMunchkin __ack_ack [2264770826798969496 7102317654394246142]
2014-06-07 06:43:27 b.s.util [ERROR] Async loop died!
java.lang.RuntimeException: java.lang.RuntimeException: Client is being closed, and does not take requests any more
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:107) ~[storm-core-0.9.1.0-incubating.jar:0.9.1.0-incubating]
at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:78) ~[storm-core-0.9.1.0-incubating.jar:0.9.1.0-incubating]
at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:77) ~[storm-core-0.9.1.0-incubating.jar:0.9.1.0-incubating]
at backtype.storm.disruptor$consume_loop_STAR_$fn__1577.invoke(disruptor.clj:89) ~[na:na]
at backtype.storm.util$async_loop$fn__384.invoke(util.clj:433) ~[na:na]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55]
Caused by: java.lang.RuntimeException: Client is being closed, and does not take requests any more
at backtype.storm.messaging.netty.Client.send(Client.java:125) ~[storm-core-0.9.1.0-incubating.jar:0.9.1.0-incubating]
at backtype.storm.daemon.worker$mk_transfer_tuples_handler$fn__4398$fn__4399.invoke(worker.clj:319) ~[na:na]
at backtype.storm.daemon.worker$mk_transfer_tuples_handler$fn__4398.invoke(worker.clj:308) ~[na:na]
at backtype.storm.disruptor$clojure_handler$reify__1560.onEvent(disruptor.clj:58) ~[na:na]
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:104) ~[storm-core-0.9.1.0-incubating.jar:0.9.1.0-incubating]
... 6 common frames omitted
2014-06-07 06:43:27 b.s.util [INFO] Halting process: ("Async loop died!")
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: Fixing an integer overflow in getSle...
Posted by easyreal <gi...@git.apache.org>.
Github user easyreal commented on the pull request:
https://github.com/apache/incubator-storm/pull/41#issuecomment-39538772
it does not resolve the underlying problems:"netty reconnect"：
https://issues.apache.org/jira/browse/STORM-187
I want to know what could cause this problem,because i found the same issue in our project
thank you
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: Fixing an integer overflow in getSle...
Posted by phoenixhadoop <gi...@git.apache.org>.
Github user phoenixhadoop commented on the pull request:
https://github.com/apache/incubator-storm/pull/41#issuecomment-39543733
good +1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: Fixing an integer overflow in getSle...
Posted by YxAc <gi...@git.apache.org>.
Github user YxAc commented on the pull request:
https://github.com/apache/incubator-storm/pull/41#issuecomment-39542762
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: Fixing an integer overflow in getSle...
Posted by d2r <gi...@git.apache.org>.
Github user d2r commented on the pull request:
https://github.com/apache/incubator-storm/pull/41#issuecomment-36747258
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: Fixing an integer overflow in getSle...
Posted by ptgoetz <gi...@git.apache.org>.
Github user ptgoetz commented on the pull request:
https://github.com/apache/incubator-storm/pull/41#issuecomment-36782030
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
Re: [GitHub] incubator-storm pull request: Fixing an integer overflow in getSle...
Posted by "Philip (flip) Kromer" <fl...@infochimps.com>.
+1
[GitHub] incubator-storm pull request: Fixing an integer overflow in getSle...
Posted by d2r <gi...@git.apache.org>.
Github user d2r commented on the pull request:
https://github.com/apache/incubator-storm/pull/41#issuecomment-41187243
@SoAG, In what version are you seeing this happen?
This should be fixed for 0.9.2.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: Fixing an integer overflow in getSle...
Posted by nathanmarz <gi...@git.apache.org>.
Github user nathanmarz commented on the pull request:
https://github.com/apache/incubator-storm/pull/41#issuecomment-36554990
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: Fixing an integer overflow in getSle...
Posted by SoAG <gi...@git.apache.org>.
Github user SoAG commented on the pull request:
https://github.com/apache/incubator-storm/pull/41#issuecomment-41374756
Hi @d2r,
We are currently using 0.9.1.
Cool that is good to know.
Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: Fixing an integer overflow in getSle...
Posted by easyreal <gi...@git.apache.org>.
Github user easyreal commented on the pull request:
https://github.com/apache/incubator-storm/pull/41#issuecomment-39801468
Thank you for your reply, I am ready to use zeromq instead of netty.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: Fixing an integer overflow in getSle...
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on the pull request:
https://github.com/apache/incubator-storm/pull/41#issuecomment-39602760
@easyreal I totally agree this does not fix connections disappearing, it just prevents the worker from crashing in some situations after a connection has gone down. There are numerous reasons why a connection can disappear. Many of them are valid reasons, like a the worker on the other end crashed. In those cases the goal is to provide a way to recover quickly and continue processing with as little interruption as possible.
If you have a reproducible use case you can post where the connections are dropped I would be very happy to help debug/fix it. However, because the connection itself is handled by netty it may be a bug in netty, or possibly something with the network your topology is running on. I think zeromq is more quite about recovering lost connections.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: Fixing an integer overflow in getSle...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-storm/pull/41
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: Fixing an integer overflow in getSle...
Posted by SoAG <gi...@git.apache.org>.
Github user SoAG commented on the pull request:
https://github.com/apache/incubator-storm/pull/41#issuecomment-41181568
Is there any news on this issue? I am encountering the same problem in my setup. After stopping one of the supervisor by killing the tasks and executors are rebalanced properly to the remaining supervisor. However restarting the the supervisor on the same instance where the supervisor was stopped results in this exception:
```
2014-04-23 15:57:56 STDIO [ERROR] Apr 23, 2014 3:57:56 PM org.jboss.netty.channel.DefaultChannelPipeline
WARNING: An exception was thrown by a user handler while handling an exception event ([id: 0x3eb4948c] EXCEPTION: java.net.ConnectException: Connection refused)
java.lang.IllegalArgumentException: timeout value is negative
at java.lang.Thread.sleep(Native Method)
at backtype.storm.messaging.netty.Client.reconnect(Client.java:94)
at backtype.storm.messaging.netty.StormClientHandler.exceptionCaught(StormClientHandler.java:118)
at org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377)
at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:525)
at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:109)
at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:78)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:41)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---