Fan-in: 120 executor bolt feeding 10 executor bolt via fieldsGrouping?

[John Yost <so...@gmail.com>]

Hi Everyone,

Recently fixed  throughput problem using localOrShuffleGrouping where I was
emitting tuples from Bolt A (400 executors) to Bolt B (would like 10
executors, but switched to 120 to ensure local messaging in 120 worker
topology). This redesign was further needed because, for each tuple
incoming to Bolt A, Bolt A generates 20 tuples, which therefore actually
results in a 8000 -> 120/10 fan in.

Next issue: I ultimately need to emit tuples from the 120 executor Bolt B
via fieldsGrouping, either via sending to Bolt C with 10 executors, or
perhaps out to Kafka via KafkaBolt and then have another topology do the
rest of the work. So the fan in is not as dramatic (120 -> 10 vs. 8000 ->
10), but not sure if this pattern will work in Storm.

Question: is it practical to use fieldsGrouping for a 120 -> 10 fan-in?
I'd really appreciate guidance on this one--thanks!

--John