You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by Chitrak Ojha <ch...@reflektion.com> on 2014/05/08 02:39:21 UTC

Storm with 1 worker configured creates two instances of workers

Hi,

We're using Storm - *0.9.1*

We have a very simple topology with 1 worker configured.

*1. We have one spout and one bolt: Here's the topology: *
    builder.setSpout("atos3_spout", AtoS3Spout(kafka_client, group, topic,
output_fields, transform, offsets), partitions)
    builder.setBolt("atos3_bolt", AtoS3Bolt(output_fields, temp_dir),
partitions).shuffleGrouping("atos3_spout")

*2. Here's the request received by Storm: *
2014-05-07 02:56:21 b.s.d.nimbus [INFO] Received topology submission for
archive_to_s3 with conf {"storm.id" "archive_to_s3-17-1399431381",
"nimbus.host" "localhost", "topology.acker.executors" nil,
"topology.kryo.decorators" (), "topology.name" "archive_to_s3",
"topology.debug" true, "topology.kryo.register" nil, *"topology.workers" 1*,
"supervisor.worker.timeout.secs" 60, "topology.max.task.parallelism" nil,
"petrel.host" "ip-10-234-21-24", "petrel.user" "rfkadmin"}


3. When we goto the *stormdist* directory to monitor log files, I see two
log files both bolt and spout. We see that the two bolts and two spouts are
separate processes.

4. Our spout is a kafka consumer. When we send a message to kafka, we see
that the message is received by both spout instances. The message is then
sent to both the bolts. Is this behavior expected since we've configured
only one worker process and with shuffle grouping, the messages should be
sprayed to bolt instances?

Also, please let us know if you have further questions.

Any help in this regard will be helpful.

Best,
Chitrak