Configurations for Multiple Clusters

[Eli Finkelshteyn <ie...@gmail.com>]

Hi,
I've recently run into a situation where I'll have access to my local 
single node cluster, and development cluster, and a production cluster. 
What's the best way to work with all three of these, easily switching 
between which I want to use? I found this project 
<http://code.google.com/p/hadoopenv/wiki/README> that seems like it's 
trying to address my issue, but it hasn't been updated since 2010. Is 
this still the preferred way of managing multiple clusters, or is there 
something better (preferably something easy to install, as I'll need to 
convince others at my company to use whatever solution I find).

Thanks!
Eli

Re: Configurations for Multiple Clusters

[Harsh J <ha...@cloudera.com>]

Eli,

Couple of things I've done (Not necessarily ideal):

* Use a git repo for configs with branches. Switch when needed.
* Point HADOOP_CONF_DIR to the right conf-dir location (or have
triggers that do these for you). Similar to above I guess.
* Alias "hadoop" multiple times to use different --config <conf dir>
opts, and use convenient aliased names.

Taking a brief look at hadoopenv sources (Last update seems to be mid
2011), I think it would work with the stable versions of Hadoop even
today. You may give it a whirl too if its approach attracts you.

On Mon, Mar 26, 2012 at 9:30 PM, Eli Finkelshteyn <ie...@gmail.com> wrote:
> Hi,
> I've recently run into a situation where I'll have access to my local single
> node cluster, and development cluster, and a production cluster. What's the
> best way to work with all three of these, easily switching between which I
> want to use? I found this project
> <http://code.google.com/p/hadoopenv/wiki/README> that seems like it's trying
> to address my issue, but it hasn't been updated since 2010. Is this still
> the preferred way of managing multiple clusters, or is there something
> better (preferably something easy to install, as I'll need to convince
> others at my company to use whatever solution I find).
>
> Thanks!
> Eli



-- 
Harsh J