You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Mark Tomko <mt...@broadinstitute.org> on 2015/04/15 18:08:19 UTC
Workaround until https://issues.apache.org/jira/browse/STORM-573 is
fixed in a release
Hi,
I'm working on some tests for one of my Storm topologies, and I'm
consistently seeing topology timeouts from the testing framework, and it
looks like it's only waiting 5 seconds, which is pretty aggressive in my
opinion:
Error in cluster
java.lang.AssertionError: Test timed out (5000ms)
at backtype.storm.testing$complete_topology.doInvoke(testing.clj:489)
~[storm-core-0.9.3.jar:0.9.3]
at clojure.lang.RestFn.invoke(RestFn.java:826) ~[clojure-1.5.1.jar:na]
at backtype.storm.testing4j$_completeTopology.invoke(testing4j.clj:61)
~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.Testing.completeTopology(Unknown Source)
[storm-core-0.9.3.jar:0.9.3]
I know from experience that this topology runs pretty well on our remote
cluster, so I don't think these timeouts are necessarily and indication of
something bad.
Looking around, I found this JIRA issue:
https://issues.apache.org/jira/browse/STORM-573
Which was fixed ~5 months ago, but didn't end up in Storm 0.9.4. Does
anyone have any suggestions for a testing workaround in the meantime?
Before I'd discovered the testing framework, I'd been just been spinning up
a local cluster myself as part of the test, and I might just go back to
doing that, but it would be great to use the officially supported testing
mechanisms.
Thanks,
Mark
Re: Workaround until https://issues.apache.org/jira/browse/STORM-573
is fixed in a release
Posted by 임정택 <ka...@gmail.com>.
You can refer https://github.com/apache/storm/pull/308 and below code.
(def TEST-TIMEOUT-MS
(let [timeout (System/getenv "STORM_TEST_TIMEOUT_MS")]
(parse-int (if timeout timeout "5000"))))
Cause of some reasons, it relies system environment variable, not JVM
property.
2015-04-17 2:33 GMT+09:00 Mark Tomko <mt...@broadinstitute.org>:
> Setting the property using System.setProperty does not work, but setting
> the property in the shell does seem to.
>
> One thing that I notice is that if a worker process dies, it kills the
> entire JVM. I'm running my tests from SBT at the moment, and when the
> exception is thrown, I'm getting dropped back to the shell, even with a
> try/catch block around the call to
> Testing.withSimulatedTimeLocalCluster(...).
>
> We could/should add some additional exception handling to the actual
> components, but I'm not sure it makes sense to kill the entire JVM just for
> an exception.
>
> Thanks!
> Mark
>
> On Thu, Apr 16, 2015 at 9:00 AM, 임정택 <ka...@gmail.com> wrote:
>
>> Hello.
>>
>> You can try set system environment "STORM_TEST_TIMEOUT_MS" and run your
>> test.
>> It'll bind to TEST_TIMEOUT_MS and it's default timeout value of
>> complete-topology.
>>
>> Hope this helps.
>>
>> Regards.
>> Jungtaek Lim (HeartSaVioR)
>>
>> 2015-04-16 1:08 GMT+09:00 Mark Tomko <mt...@broadinstitute.org>:
>>
>>> Hi,
>>>
>>> I'm working on some tests for one of my Storm topologies, and I'm
>>> consistently seeing topology timeouts from the testing framework, and it
>>> looks like it's only waiting 5 seconds, which is pretty aggressive in my
>>> opinion:
>>>
>>> Error in cluster
>>> java.lang.AssertionError: Test timed out (5000ms)
>>> at backtype.storm.testing$complete_topology.doInvoke(testing.clj:489)
>>> ~[storm-core-0.9.3.jar:0.9.3]
>>> at clojure.lang.RestFn.invoke(RestFn.java:826) ~[clojure-1.5.1.jar:na]
>>> at backtype.storm.testing4j$_completeTopology.invoke(testing4j.clj:61)
>>> ~[storm-core-0.9.3.jar:0.9.3]
>>> at backtype.storm.Testing.completeTopology(Unknown Source)
>>> [storm-core-0.9.3.jar:0.9.3]
>>>
>>>
>>> I know from experience that this topology runs pretty well on our remote
>>> cluster, so I don't think these timeouts are necessarily and indication of
>>> something bad.
>>>
>>> Looking around, I found this JIRA issue:
>>>
>>> https://issues.apache.org/jira/browse/STORM-573
>>>
>>> Which was fixed ~5 months ago, but didn't end up in Storm 0.9.4. Does
>>> anyone have any suggestions for a testing workaround in the meantime?
>>> Before I'd discovered the testing framework, I'd been just been spinning up
>>> a local cluster myself as part of the test, and I might just go back to
>>> doing that, but it would be great to use the officially supported testing
>>> mechanisms.
>>>
>>> Thanks,
>>>
>>> Mark
>>>
>>>
>>
>>
>> --
>> Name : 임 정택
>> Blog : http://www.heartsavior.net / http://dev.heartsavior.net
>> Twitter : http://twitter.com/heartsavior
>> LinkedIn : http://www.linkedin.com/in/heartsavior
>>
>
>
--
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior
Re: Workaround until https://issues.apache.org/jira/browse/STORM-573
is fixed in a release
Posted by Mark Tomko <mt...@broadinstitute.org>.
Setting the property using System.setProperty does not work, but setting
the property in the shell does seem to.
One thing that I notice is that if a worker process dies, it kills the
entire JVM. I'm running my tests from SBT at the moment, and when the
exception is thrown, I'm getting dropped back to the shell, even with a
try/catch block around the call to
Testing.withSimulatedTimeLocalCluster(...).
We could/should add some additional exception handling to the actual
components, but I'm not sure it makes sense to kill the entire JVM just for
an exception.
Thanks!
Mark
On Thu, Apr 16, 2015 at 9:00 AM, 임정택 <ka...@gmail.com> wrote:
> Hello.
>
> You can try set system environment "STORM_TEST_TIMEOUT_MS" and run your
> test.
> It'll bind to TEST_TIMEOUT_MS and it's default timeout value of
> complete-topology.
>
> Hope this helps.
>
> Regards.
> Jungtaek Lim (HeartSaVioR)
>
> 2015-04-16 1:08 GMT+09:00 Mark Tomko <mt...@broadinstitute.org>:
>
>> Hi,
>>
>> I'm working on some tests for one of my Storm topologies, and I'm
>> consistently seeing topology timeouts from the testing framework, and it
>> looks like it's only waiting 5 seconds, which is pretty aggressive in my
>> opinion:
>>
>> Error in cluster
>> java.lang.AssertionError: Test timed out (5000ms)
>> at backtype.storm.testing$complete_topology.doInvoke(testing.clj:489)
>> ~[storm-core-0.9.3.jar:0.9.3]
>> at clojure.lang.RestFn.invoke(RestFn.java:826) ~[clojure-1.5.1.jar:na]
>> at backtype.storm.testing4j$_completeTopology.invoke(testing4j.clj:61)
>> ~[storm-core-0.9.3.jar:0.9.3]
>> at backtype.storm.Testing.completeTopology(Unknown Source)
>> [storm-core-0.9.3.jar:0.9.3]
>>
>>
>> I know from experience that this topology runs pretty well on our remote
>> cluster, so I don't think these timeouts are necessarily and indication of
>> something bad.
>>
>> Looking around, I found this JIRA issue:
>>
>> https://issues.apache.org/jira/browse/STORM-573
>>
>> Which was fixed ~5 months ago, but didn't end up in Storm 0.9.4. Does
>> anyone have any suggestions for a testing workaround in the meantime?
>> Before I'd discovered the testing framework, I'd been just been spinning up
>> a local cluster myself as part of the test, and I might just go back to
>> doing that, but it would be great to use the officially supported testing
>> mechanisms.
>>
>> Thanks,
>>
>> Mark
>>
>>
>
>
> --
> Name : 임 정택
> Blog : http://www.heartsavior.net / http://dev.heartsavior.net
> Twitter : http://twitter.com/heartsavior
> LinkedIn : http://www.linkedin.com/in/heartsavior
>
Re: Workaround until https://issues.apache.org/jira/browse/STORM-573
is fixed in a release
Posted by 임정택 <ka...@gmail.com>.
Hello.
You can try set system environment "STORM_TEST_TIMEOUT_MS" and run your
test.
It'll bind to TEST_TIMEOUT_MS and it's default timeout value of
complete-topology.
Hope this helps.
Regards.
Jungtaek Lim (HeartSaVioR)
2015-04-16 1:08 GMT+09:00 Mark Tomko <mt...@broadinstitute.org>:
> Hi,
>
> I'm working on some tests for one of my Storm topologies, and I'm
> consistently seeing topology timeouts from the testing framework, and it
> looks like it's only waiting 5 seconds, which is pretty aggressive in my
> opinion:
>
> Error in cluster
> java.lang.AssertionError: Test timed out (5000ms)
> at backtype.storm.testing$complete_topology.doInvoke(testing.clj:489)
> ~[storm-core-0.9.3.jar:0.9.3]
> at clojure.lang.RestFn.invoke(RestFn.java:826) ~[clojure-1.5.1.jar:na]
> at backtype.storm.testing4j$_completeTopology.invoke(testing4j.clj:61)
> ~[storm-core-0.9.3.jar:0.9.3]
> at backtype.storm.Testing.completeTopology(Unknown Source)
> [storm-core-0.9.3.jar:0.9.3]
>
>
> I know from experience that this topology runs pretty well on our remote
> cluster, so I don't think these timeouts are necessarily and indication of
> something bad.
>
> Looking around, I found this JIRA issue:
>
> https://issues.apache.org/jira/browse/STORM-573
>
> Which was fixed ~5 months ago, but didn't end up in Storm 0.9.4. Does
> anyone have any suggestions for a testing workaround in the meantime?
> Before I'd discovered the testing framework, I'd been just been spinning up
> a local cluster myself as part of the test, and I might just go back to
> doing that, but it would be great to use the officially supported testing
> mechanisms.
>
> Thanks,
>
> Mark
>
>
--
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior