how to specify hardware-id per whirr instance template

[Satish Mittal <sa...@gmail.com>]

Hi All,

I recently tried out apache whirr to spin off EC2 instances and deploy my
stack - that comprises of hadoop DN/NN/TT/JT and my distributed
applications that use hadoop. It worked fine. However in my recipe file, I
could give only one hardware-id:

whirr.hardware-id=m1.medium

However this configuration applies to all instance-templates in my recipe.
Ideally I would like to specify different hardwares for different instance
templates - so that I could spin off EC2 instance type depending upon the
combination of processes specified in the template (larger machine for
large number of processes, and vice-versa). Is it possible to do so
currently? I tried looking into the doc available, but couldn't find any
way. Am I missing anything?

Thanks,
Satish

-- 
"The happiest of people don´t necessarily have the best of everything, they
just make the most of everything that comes along their way."

Re: how to specify hardware-id per whirr instance template

[Andrew Bayer <an...@gmail.com>]

Actually, you keep the whirr.instance-templates line and add the
whirr.templates lines.

"whirr.templates.hadoop-namenode+A.hardware-id=m1.large" means "When
setting up the instances specified in whirr.instance-templates for the role
combo hadoop-namenode+A, use this hardware ID". It's an extra level of
configuration on top of the instance templates, not replacing them.

Hmm. I don't think we've got anything that currently supports running
multiple instances of a role on the same node. It could probably be done,
but would take some work in the core and a service that was explicitly
designed for that so that one instance didn't stomp on another...

A.

On Wed, Aug 7, 2013 at 11:41 AM, Satish Mittal <sa...@gmail.com>wrote:

> Thanks Andrew!
>
> So to confirm, currently I am specifying 2 nodes:
> whirr.instance-templates=1 hadoop-namenode+A, 1 hadoop-datanode+B
>
> now I need to replace it with 2 separate entries:
> whirr.templates.hadoop-namenode+A.hardware-id=m1.large
> whirr.templates.hadoop-datanode+B.hardware-id=m1.medium
>
> Is this correct?
>
> Also, a follow-up question: suppose I want to specify x instance of A and
> y instances of B on the same node. How can I do that? i.e. In general, can
> we specify separate number for individual roles on an instance?
>
> -Satish
>
>
> On Wed, Aug 7, 2013 at 11:41 PM, Andrew Bayer <an...@gmail.com>wrote:
>
>> Yup, there's an easy way to do that -
>>
>> whirr.templates.foo+bar.hardware-id=m1.large or whatever, where "foo+bar"
>> is the role/roles to use that hardware ID.
>>
>> A.
>>
>>
>> On Wed, Aug 7, 2013 at 10:56 AM, Satish Mittal <sa...@gmail.com>wrote:
>>
>>> Hi All,
>>>
>>> I recently tried out apache whirr to spin off EC2 instances and deploy
>>> my stack - that comprises of hadoop DN/NN/TT/JT and my distributed
>>> applications that use hadoop. It worked fine. However in my recipe file, I
>>> could give only one hardware-id:
>>>
>>> whirr.hardware-id=m1.medium
>>>
>>> However this configuration applies to all instance-templates in my
>>> recipe. Ideally I would like to specify different hardwares for different
>>> instance templates - so that I could spin off EC2 instance type depending
>>> upon the combination of processes specified in the template (larger machine
>>> for large number of processes, and vice-versa). Is it possible to do so
>>> currently? I tried looking into the doc available, but couldn't find any
>>> way. Am I missing anything?
>>>
>>> Thanks,
>>> Satish
>>>
>>> --
>>> "The happiest of people don´t necessarily have the best of everything,
>>> they just make the most of everything that comes along their way."
>>>
>>
>>
>
>
> --
> "The happiest of people don´t necessarily have the best of everything,
> they just make the most of everything that comes along their way."
>

Re: how to specify hardware-id per whirr instance template

[Satish Mittal <sa...@gmail.com>]

Thanks Andrew!

So to confirm, currently I am specifying 2 nodes:
whirr.instance-templates=1 hadoop-namenode+A, 1 hadoop-datanode+B

now I need to replace it with 2 separate entries:
whirr.templates.hadoop-namenode+A.hardware-id=m1.large
whirr.templates.hadoop-datanode+B.hardware-id=m1.medium

Is this correct?

Also, a follow-up question: suppose I want to specify x instance of A and y
instances of B on the same node. How can I do that? i.e. In general, can we
specify separate number for individual roles on an instance?

-Satish

On Wed, Aug 7, 2013 at 11:41 PM, Andrew Bayer <an...@gmail.com>wrote:

> Yup, there's an easy way to do that -
>
> whirr.templates.foo+bar.hardware-id=m1.large or whatever, where "foo+bar"
> is the role/roles to use that hardware ID.
>
> A.
>
>
> On Wed, Aug 7, 2013 at 10:56 AM, Satish Mittal <sa...@gmail.com>wrote:
>
>> Hi All,
>>
>> I recently tried out apache whirr to spin off EC2 instances and deploy my
>> stack - that comprises of hadoop DN/NN/TT/JT and my distributed
>> applications that use hadoop. It worked fine. However in my recipe file, I
>> could give only one hardware-id:
>>
>> whirr.hardware-id=m1.medium
>>
>> However this configuration applies to all instance-templates in my
>> recipe. Ideally I would like to specify different hardwares for different
>> instance templates - so that I could spin off EC2 instance type depending
>> upon the combination of processes specified in the template (larger machine
>> for large number of processes, and vice-versa). Is it possible to do so
>> currently? I tried looking into the doc available, but couldn't find any
>> way. Am I missing anything?
>>
>> Thanks,
>> Satish
>>
>> --
>> "The happiest of people don´t necessarily have the best of everything,
>> they just make the most of everything that comes along their way."
>>
>
>


-- 
"The happiest of people don´t necessarily have the best of everything, they
just make the most of everything that comes along their way."

Re: how to specify hardware-id per whirr instance template

[Andrew Bayer <an...@gmail.com>]

Yup, there's an easy way to do that -

whirr.templates.foo+bar.hardware-id=m1.large or whatever, where "foo+bar"
is the role/roles to use that hardware ID.

A.

On Wed, Aug 7, 2013 at 10:56 AM, Satish Mittal <sa...@gmail.com>wrote:

> Hi All,
>
> I recently tried out apache whirr to spin off EC2 instances and deploy my
> stack - that comprises of hadoop DN/NN/TT/JT and my distributed
> applications that use hadoop. It worked fine. However in my recipe file, I
> could give only one hardware-id:
>
> whirr.hardware-id=m1.medium
>
> However this configuration applies to all instance-templates in my recipe.
> Ideally I would like to specify different hardwares for different instance
> templates - so that I could spin off EC2 instance type depending upon the
> combination of processes specified in the template (larger machine for
> large number of processes, and vice-versa). Is it possible to do so
> currently? I tried looking into the doc available, but couldn't find any
> way. Am I missing anything?
>
> Thanks,
> Satish
>
> --
> "The happiest of people don´t necessarily have the best of everything,
> they just make the most of everything that comes along their way."
>