Multiple Storm workers on one machine--throughput and need for custom scheduler?

[John Yost <ho...@gmail.com>]

Hi Everyone,

I am running a Storm topology in a highly multi-tenant environment--both in
general and in terms of Storm--where there are two other Storm topologies
running on the same Storm Mesos cluster.

I am observing that the executors for the worker co-located with another
worker tend to have sharply lower throughput as measured by number of
tuples acked at the DAG entry point (KafkaSpout).

I remember reading that it's best to have one Storm worker per physical
node. I imagine there is a way to write a custom scheduler to filter out
nodes that already have a Storm worker executing. Does this make sense, or
are there ways to make our worker nodes more amenable to 1..n workers?

Any ideas/thoughts very much appreciated--thanks! :)

--John