You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Andrei Savu (Created) (JIRA)" <ji...@apache.org> on 2012/01/02 15:18:30 UTC
[jira] [Created] (WHIRR-461) Allow user to specify spot instance
price per instance template group
Allow user to specify spot instance price per instance template group
---------------------------------------------------------------------
Key: WHIRR-461
URL: https://issues.apache.org/jira/browse/WHIRR-461
Project: Whirr
Issue Type: Improvement
Reporter: Andrei Savu
Assignee: Andrei Savu
Fix For: 0.8.0
A natural improvement on top of WHIRR-332 is to allow the user to specify the spot instance price per instance template group.
Proposed syntax:
{noformat}
whirr.templates.role1+role2.aws-ec2-spot-price=0.23
{noformat}
Using this feature we can have a Hadoop cluster that has both regular and spot instance like this:
{noformat}
whirr.cluster-name=hadoop-on-ec2
whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker, 3 hadoop-datanode+hadoop-jobtracker, 7 hadoop-datanode+hadoop-jobtracker+noop
# user a large instance for the namenode as needed
whirr.templates.hadoop-namenode+hadoop-jobtracker.hardware-id=c1.xlarge
# regular smaller instance for a few data nodes
whirr.templates.hadoop-datanode+hadoop-jobtracker.hardware-id=m1.large
# and spot instances for the rest of the datanodes
whirr.templats.hadoop-datanode+hadoop-tasktracker+noop.aws-ec2-spot-price=0.32
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-461) Allow user to specify spot instance
price per instance template group
Posted by "Andrei Savu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-461:
------------------------------
Attachment: WHIRR-461.patch
Patch changes:
* cleanup InstanceTemplate + support for spot price
* unit tests
I have done some manual testing on aws-ec2 and everything seems to be working fine.
> Allow user to specify spot instance price per instance template group
> ---------------------------------------------------------------------
>
> Key: WHIRR-461
> URL: https://issues.apache.org/jira/browse/WHIRR-461
> Project: Whirr
> Issue Type: Improvement
> Reporter: Andrei Savu
> Assignee: Andrei Savu
> Fix For: 0.8.0
>
> Attachments: WHIRR-461.patch
>
>
> A natural improvement on top of WHIRR-332 is to allow the user to specify the spot instance price per instance template group.
> Proposed syntax:
> {noformat}
> whirr.templates.role1+role2.aws-ec2-spot-price=0.23
> {noformat}
> Using this feature we can have a Hadoop cluster that has both regular and spot instance like this:
> {noformat}
> whirr.cluster-name=hadoop-on-ec2
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker, 3 hadoop-datanode+hadoop-jobtracker, 7 hadoop-datanode+hadoop-jobtracker+noop
> # user a large instance for the namenode as needed
> whirr.templates.hadoop-namenode+hadoop-jobtracker.hardware-id=c1.xlarge
> # regular smaller instance for a few data nodes
> whirr.templates.hadoop-datanode+hadoop-jobtracker.hardware-id=m1.large
> # and spot instances for the rest of the datanodes
> whirr.templats.hadoop-datanode+hadoop-tasktracker+noop.aws-ec2-spot-price=0.32
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-461) Allow user to specify spot instance
price per instance template group
Posted by "Adrian Cole (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180660#comment-13180660 ]
Adrian Cole commented on WHIRR-461:
-----------------------------------
okie then +1 as my comment is more general than this change
> Allow user to specify spot instance price per instance template group
> ---------------------------------------------------------------------
>
> Key: WHIRR-461
> URL: https://issues.apache.org/jira/browse/WHIRR-461
> Project: Whirr
> Issue Type: Improvement
> Reporter: Andrei Savu
> Assignee: Andrei Savu
> Fix For: 0.8.0
>
> Attachments: WHIRR-461.patch
>
>
> A natural improvement on top of WHIRR-332 is to allow the user to specify the spot instance price per instance template group.
> Proposed syntax:
> {noformat}
> whirr.templates.role1+role2.aws-ec2-spot-price=0.23
> {noformat}
> Using this feature we can have a Hadoop cluster that has both regular and spot instance like this:
> {noformat}
> whirr.cluster-name=hadoop-on-ec2
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker, 3 hadoop-datanode+hadoop-jobtracker, 7 hadoop-datanode+hadoop-jobtracker+noop
> # user a large instance for the namenode as needed
> whirr.templates.hadoop-namenode+hadoop-jobtracker.hardware-id=c1.xlarge
> # regular smaller instance for a few data nodes
> whirr.templates.hadoop-datanode+hadoop-jobtracker.hardware-id=m1.large
> # and spot instances for the rest of the datanodes
> whirr.templats.hadoop-datanode+hadoop-tasktracker+noop.aws-ec2-spot-price=0.32
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-461) Allow user to specify spot instance
price per instance template group
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180656#comment-13180656 ]
Andrei Savu commented on WHIRR-461:
-----------------------------------
I'm using aws-ec2-spot-price for consistency with whirr.aws-ec2-spot-price.
> Allow user to specify spot instance price per instance template group
> ---------------------------------------------------------------------
>
> Key: WHIRR-461
> URL: https://issues.apache.org/jira/browse/WHIRR-461
> Project: Whirr
> Issue Type: Improvement
> Reporter: Andrei Savu
> Assignee: Andrei Savu
> Fix For: 0.8.0
>
> Attachments: WHIRR-461.patch
>
>
> A natural improvement on top of WHIRR-332 is to allow the user to specify the spot instance price per instance template group.
> Proposed syntax:
> {noformat}
> whirr.templates.role1+role2.aws-ec2-spot-price=0.23
> {noformat}
> Using this feature we can have a Hadoop cluster that has both regular and spot instance like this:
> {noformat}
> whirr.cluster-name=hadoop-on-ec2
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker, 3 hadoop-datanode+hadoop-jobtracker, 7 hadoop-datanode+hadoop-jobtracker+noop
> # user a large instance for the namenode as needed
> whirr.templates.hadoop-namenode+hadoop-jobtracker.hardware-id=c1.xlarge
> # regular smaller instance for a few data nodes
> whirr.templates.hadoop-datanode+hadoop-jobtracker.hardware-id=m1.large
> # and spot instances for the rest of the datanodes
> whirr.templats.hadoop-datanode+hadoop-tasktracker+noop.aws-ec2-spot-price=0.32
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-461) Allow user to specify spot instance
price per instance template group
Posted by "Adrian Cole (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180644#comment-13180644 ]
Adrian Cole commented on WHIRR-461:
-----------------------------------
looks good, except I'd prefer clearer scoping on custom properties like spot price to use the provider id:
instead of: aws-ec2-spot-price, aws-ec2.spot-price
> Allow user to specify spot instance price per instance template group
> ---------------------------------------------------------------------
>
> Key: WHIRR-461
> URL: https://issues.apache.org/jira/browse/WHIRR-461
> Project: Whirr
> Issue Type: Improvement
> Reporter: Andrei Savu
> Assignee: Andrei Savu
> Fix For: 0.8.0
>
> Attachments: WHIRR-461.patch
>
>
> A natural improvement on top of WHIRR-332 is to allow the user to specify the spot instance price per instance template group.
> Proposed syntax:
> {noformat}
> whirr.templates.role1+role2.aws-ec2-spot-price=0.23
> {noformat}
> Using this feature we can have a Hadoop cluster that has both regular and spot instance like this:
> {noformat}
> whirr.cluster-name=hadoop-on-ec2
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker, 3 hadoop-datanode+hadoop-jobtracker, 7 hadoop-datanode+hadoop-jobtracker+noop
> # user a large instance for the namenode as needed
> whirr.templates.hadoop-namenode+hadoop-jobtracker.hardware-id=c1.xlarge
> # regular smaller instance for a few data nodes
> whirr.templates.hadoop-datanode+hadoop-jobtracker.hardware-id=m1.large
> # and spot instances for the rest of the datanodes
> whirr.templats.hadoop-datanode+hadoop-tasktracker+noop.aws-ec2-spot-price=0.32
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira