You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Andrei Savu (JIRA)" <ji...@apache.org> on 2011/05/30 17:44:47 UTC
[jira] [Created] (WHIRR-318) More extensible cluster spec file
format
More extensible cluster spec file format
----------------------------------------
Key: WHIRR-318
URL: https://issues.apache.org/jira/browse/WHIRR-318
Project: Whirr
Issue Type: Improvement
Reporter: Andrei Savu
We are slowly getting to the point were our ability to improve Whirr will be limited by the cluster spec format. I'm opening this issue so that we can start to think about a new format that it's going to be more flexible.
Some of the characteristics of the new format should be:
* low verbosity and easily editable using standard text editors. (e.g yaml)
* easy to parse using existing libraries (e.g. snakeyaml)
* should allow values to be overridden as CLI parameters (e.g yaml liniarization - elasticsearch is doing this).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (WHIRR-318) More extensible cluster spec file
format
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu reassigned WHIRR-318:
---------------------------------
Assignee: Andrei Savu
> More extensible cluster spec file format
> ----------------------------------------
>
> Key: WHIRR-318
> URL: https://issues.apache.org/jira/browse/WHIRR-318
> Project: Whirr
> Issue Type: Improvement
> Reporter: Andrei Savu
> Assignee: Andrei Savu
> Attachments: hadoop.yml, new-format.yaml
>
>
> We are slowly getting to the point were our ability to improve Whirr will be limited by the cluster spec format. I'm opening this issue so that we can start to think about a new format that it's going to be more flexible.
> Some of the characteristics of the new format should be:
> * low verbosity and easily editable using standard text editors. (e.g yaml)
> * easy to parse using existing libraries (e.g. snakeyaml)
> * should allow values to be overridden as CLI parameters (e.g yaml liniarization - elasticsearch is doing this).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-318) More extensible cluster spec file
format
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041412#comment-13041412 ]
Tom White commented on WHIRR-318:
---------------------------------
This seems like a good direction to me. I assume that it would be an alternative to properties file.
Rather than carrying over the role1+role2 notation, which can get awkward, it would be better to use a more natural YAML syntax (or JSON, if we use that). Something like
{noformat}
templates:
hadoop-master:
roles:
- hadoop-namenode
- hadoop-jobtracker
image-id: ...
{noformat}
> More extensible cluster spec file format
> ----------------------------------------
>
> Key: WHIRR-318
> URL: https://issues.apache.org/jira/browse/WHIRR-318
> Project: Whirr
> Issue Type: Improvement
> Reporter: Andrei Savu
> Attachments: hadoop.yml
>
>
> We are slowly getting to the point were our ability to improve Whirr will be limited by the cluster spec format. I'm opening this issue so that we can start to think about a new format that it's going to be more flexible.
> Some of the characteristics of the new format should be:
> * low verbosity and easily editable using standard text editors. (e.g yaml)
> * easy to parse using existing libraries (e.g. snakeyaml)
> * should allow values to be overridden as CLI parameters (e.g yaml liniarization - elasticsearch is doing this).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-318) More extensible cluster spec file
format
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13051260#comment-13051260 ]
Andrei Savu commented on WHIRR-318:
-----------------------------------
Guys should I move on and create a proof-of-concept implementation? It shouldn't take too much time.
> More extensible cluster spec file format
> ----------------------------------------
>
> Key: WHIRR-318
> URL: https://issues.apache.org/jira/browse/WHIRR-318
> Project: Whirr
> Issue Type: Improvement
> Reporter: Andrei Savu
> Attachments: hadoop.yml
>
>
> We are slowly getting to the point were our ability to improve Whirr will be limited by the cluster spec format. I'm opening this issue so that we can start to think about a new format that it's going to be more flexible.
> Some of the characteristics of the new format should be:
> * low verbosity and easily editable using standard text editors. (e.g yaml)
> * easy to parse using existing libraries (e.g. snakeyaml)
> * should allow values to be overridden as CLI parameters (e.g yaml liniarization - elasticsearch is doing this).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-318) More extensible cluster spec file
format
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13052351#comment-13052351 ]
Tom White commented on WHIRR-318:
---------------------------------
Looks good. Do you agree that we should continue to allow specification by properties files too? It should be possible to map one format to the other.
> More extensible cluster spec file format
> ----------------------------------------
>
> Key: WHIRR-318
> URL: https://issues.apache.org/jira/browse/WHIRR-318
> Project: Whirr
> Issue Type: Improvement
> Reporter: Andrei Savu
> Attachments: hadoop.yml, new-format.yaml
>
>
> We are slowly getting to the point were our ability to improve Whirr will be limited by the cluster spec format. I'm opening this issue so that we can start to think about a new format that it's going to be more flexible.
> Some of the characteristics of the new format should be:
> * low verbosity and easily editable using standard text editors. (e.g yaml)
> * easy to parse using existing libraries (e.g. snakeyaml)
> * should allow values to be overridden as CLI parameters (e.g yaml liniarization - elasticsearch is doing this).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-318) More extensible cluster spec file
format
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-318:
------------------------------
Attachment: hadoop.yml
A first iteration as described (incorporated Adrian's feedback from the email list)
> More extensible cluster spec file format
> ----------------------------------------
>
> Key: WHIRR-318
> URL: https://issues.apache.org/jira/browse/WHIRR-318
> Project: Whirr
> Issue Type: Improvement
> Reporter: Andrei Savu
> Attachments: hadoop.yml
>
>
> We are slowly getting to the point were our ability to improve Whirr will be limited by the cluster spec format. I'm opening this issue so that we can start to think about a new format that it's going to be more flexible.
> Some of the characteristics of the new format should be:
> * low verbosity and easily editable using standard text editors. (e.g yaml)
> * easy to parse using existing libraries (e.g. snakeyaml)
> * should allow values to be overridden as CLI parameters (e.g yaml liniarization - elasticsearch is doing this).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-318) More extensible cluster spec file
format
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-318:
------------------------------
Attachment: new-format.yaml
Attached a new version that incorporates Tom's feedback.
> More extensible cluster spec file format
> ----------------------------------------
>
> Key: WHIRR-318
> URL: https://issues.apache.org/jira/browse/WHIRR-318
> Project: Whirr
> Issue Type: Improvement
> Reporter: Andrei Savu
> Attachments: hadoop.yml, new-format.yaml
>
>
> We are slowly getting to the point were our ability to improve Whirr will be limited by the cluster spec format. I'm opening this issue so that we can start to think about a new format that it's going to be more flexible.
> Some of the characteristics of the new format should be:
> * low verbosity and easily editable using standard text editors. (e.g yaml)
> * easy to parse using existing libraries (e.g. snakeyaml)
> * should allow values to be overridden as CLI parameters (e.g yaml liniarization - elasticsearch is doing this).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-318) More extensible cluster spec file
format
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13052461#comment-13052461 ]
Andrei Savu commented on WHIRR-318:
-----------------------------------
bq. Do you agree that we should continue to allow specification by properties files too?
Definitely. We could consider deprecating the .properties format later (2-3 releases).
> More extensible cluster spec file format
> ----------------------------------------
>
> Key: WHIRR-318
> URL: https://issues.apache.org/jira/browse/WHIRR-318
> Project: Whirr
> Issue Type: Improvement
> Reporter: Andrei Savu
> Attachments: hadoop.yml, new-format.yaml
>
>
> We are slowly getting to the point were our ability to improve Whirr will be limited by the cluster spec format. I'm opening this issue so that we can start to think about a new format that it's going to be more flexible.
> Some of the characteristics of the new format should be:
> * low verbosity and easily editable using standard text editors. (e.g yaml)
> * easy to parse using existing libraries (e.g. snakeyaml)
> * should allow values to be overridden as CLI parameters (e.g yaml liniarization - elasticsearch is doing this).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-318) More extensible cluster spec file
format
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041290#comment-13041290 ]
Andrei Savu commented on WHIRR-318:
-----------------------------------
I like that idea of having an open source multi-cloud alternative to cloud formation.
> More extensible cluster spec file format
> ----------------------------------------
>
> Key: WHIRR-318
> URL: https://issues.apache.org/jira/browse/WHIRR-318
> Project: Whirr
> Issue Type: Improvement
> Reporter: Andrei Savu
> Attachments: hadoop.yml
>
>
> We are slowly getting to the point were our ability to improve Whirr will be limited by the cluster spec format. I'm opening this issue so that we can start to think about a new format that it's going to be more flexible.
> Some of the characteristics of the new format should be:
> * low verbosity and easily editable using standard text editors. (e.g yaml)
> * easy to parse using existing libraries (e.g. snakeyaml)
> * should allow values to be overridden as CLI parameters (e.g yaml liniarization - elasticsearch is doing this).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-318) More extensible cluster spec file
format
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041204#comment-13041204 ]
Adrian Cole commented on WHIRR-318:
-----------------------------------
might also have a look here: https://github.com/jclouds/Infrastructure-Deployment-Spec
> More extensible cluster spec file format
> ----------------------------------------
>
> Key: WHIRR-318
> URL: https://issues.apache.org/jira/browse/WHIRR-318
> Project: Whirr
> Issue Type: Improvement
> Reporter: Andrei Savu
> Attachments: hadoop.yml
>
>
> We are slowly getting to the point were our ability to improve Whirr will be limited by the cluster spec format. I'm opening this issue so that we can start to think about a new format that it's going to be more flexible.
> Some of the characteristics of the new format should be:
> * low verbosity and easily editable using standard text editors. (e.g yaml)
> * easy to parse using existing libraries (e.g. snakeyaml)
> * should allow values to be overridden as CLI parameters (e.g yaml liniarization - elasticsearch is doing this).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira