Trying to build out Storm / CDH Yarn

[Michael Segel <mi...@hotmail.com>]

Hi,

I'm still fairly new to Whirr and I am starting to run in to some issues that  I would love to get some suggestions to see if want I want is feasible…

I'm trying to use Whirr to build out a cluster on AWS using Storm , CDH 4 running YARN , HBase and Yahoo! 's storm-yarn code.

So far, I've cobbled together a recipe to build the CDH4 Yarn cluster w HBase and ZK running. 

My next task is to add my own resource file to put the storm-yarn jar on every node. 

I think I have a handle on that… 

But now I am hitting a wall… 

Need advice on how to load Storm on the cluster.
I need to create a single node as a Nimbus node, not to mention their own ZK cluster (1 or 3 nodes)

Here's the rub. Installing HBase, I run a resource script to install the ZK for HBase.
If I want to keep the storm and HBase separate is there a way to identify different ZK quorums? 

Is there a way to integrate Whirr so that it could launch a Storm script?
(Do this first in my storm-yarn resource script?)
If this works, how can I get the cluster to see the other node?

Is there any documentation on how to write a service ? 

Thx