[jira] [Commented] (STORM-2983) Some topologies not working properly

["Roshan Naik (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STORM-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387243#comment-16387243 ] 

Roshan Naik commented on STORM-2983:
------------------------------------

[~ethanli] 

I notice two issues with this setting:
{code:java}
storm.scheduler: "org.apache.storm.scheduler.resource.ResourceAwareScheduler"{code}
*1) Wrong Number of workers:*

With this setting applied, storm is spinning up the wrong number of workers for this command line:
{code:java}
bin/storm jar storm-loadgen-2.0.0-SNAPSHOT.jar  org.apache.storm.loadgen.ThroughputVsLatency  --name tvl  --spouts 1 --splitters 2 --counters 1   -c topology.workers=1{code}
It actually spins up 2 workers instead of 1. The acker is isolated into its own separate worker process while the remaining spouts and bolts sit in a single worker. 

In 2306, if worker count is 1, the inter-worker transfer thread will not be spun up.  [Reference|https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/daemon/worker/Worker.java#L219-L222].

Thats why the problem was not noticed previously.

 

*2) That setting is internally unkown.* 

Strangely, In the code I dont see any setting with the name `storm.scheduler` .  I do see something called `topology.scheduler.strategy`. Its a bit puzzling why an unknown setting makes any impact.

> Some topologies not working properly 
> -------------------------------------
>
>                 Key: STORM-2983
>                 URL: https://issues.apache.org/jira/browse/STORM-2983
>             Project: Apache Storm
>          Issue Type: Bug
>            Reporter: Ethan Li
>            Priority: Major
>
> For example,
> {code:java}
> bin/storm jar storm-loadgen-*.jar org.apache.storm.loadgen.ThroughputVsLatency --spouts 1 --splitters 2 --counters 1 -c topology.debug=true
> {code}
> on ResourceAwareScheduler not working properly.
> With default cluster settings, there will be only one __acker-executor and it will be on a separate worker. And it looks like the __acker-executor was not able to receive messages from spouts and bolts. And spouts and bolts continued to retry sending messages to acker. It then led to another problem:
> STORM-2970
> I tried to run on storm right before [https://github.com/apache/storm/pull/2502] and right after and confirmed that this bug should be related to it



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)