You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by "Roman Shaposhnik (JIRA)" <ji...@apache.org> on 2012/09/21 06:28:08 UTC
[jira] [Created] (BIGTOP-710) Create a higher level orchestration
deployment framework
Roman Shaposhnik created BIGTOP-710:
---------------------------------------
Summary: Create a higher level orchestration deployment framework
Key: BIGTOP-710
URL: https://issues.apache.org/jira/browse/BIGTOP-710
Project: Bigtop
Issue Type: Improvement
Components: Deployment
Affects Versions: 0.4.0
Reporter: Roman Shaposhnik
Assignee: Roman Shaposhnik
Fix For: 0.5.0
It would be very nice to start using our puppet code as a basis for a higher level orchestration framework capable of driving a true distributed multi-node deployment of Bigtop components. Puppet code is great at stamping out a predictable single node deployments, but we have to extend it with a DSL aimed at solving higher level orchestration.
This JIRA is meant to be a central information radiator for collecting ideas on how to approach this (what DSL to use, how to structure communication between agents and fact sharing, etc).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-710) Create a higher level orchestration
deployment framework
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460220#comment-13460220 ]
Konstantin Boudnik commented on BIGTOP-710:
-------------------------------------------
I almost feel Glu ;)
> Create a higher level orchestration deployment framework
> --------------------------------------------------------
>
> Key: BIGTOP-710
> URL: https://issues.apache.org/jira/browse/BIGTOP-710
> Project: Bigtop
> Issue Type: Improvement
> Components: Deployment
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.5.0
>
>
> It would be very nice to start using our puppet code as a basis for a higher level orchestration framework capable of driving a true distributed multi-node deployment of Bigtop components. Puppet code is great at stamping out a predictable single node deployments, but we have to extend it with a DSL aimed at solving higher level orchestration.
> This JIRA is meant to be a central information radiator for collecting ideas on how to approach this (what DSL to use, how to structure communication between agents and fact sharing, etc).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-710) Create a higher level orchestration
deployment framework
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated BIGTOP-710:
--------------------------------------
Labels: orchestration (was: )
> Create a higher level orchestration deployment framework
> --------------------------------------------------------
>
> Key: BIGTOP-710
> URL: https://issues.apache.org/jira/browse/BIGTOP-710
> Project: Bigtop
> Issue Type: Improvement
> Components: Deployment
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Labels: orchestration
> Fix For: 0.5.0
>
>
> It would be very nice to start using our puppet code as a basis for a higher level orchestration framework capable of driving a true distributed multi-node deployment of Bigtop components. Puppet code is great at stamping out a predictable single node deployments, but we have to extend it with a DSL aimed at solving higher level orchestration.
> This JIRA is meant to be a central information radiator for collecting ideas on how to approach this (what DSL to use, how to structure communication between agents and fact sharing, etc).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-710) Create a higher level orchestration
deployment framework
Posted by "Johnny Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460200#comment-13460200 ]
Johnny Zhang commented on BIGTOP-710:
-------------------------------------
@Roman, does this related to package tests in anyway ?
> Create a higher level orchestration deployment framework
> --------------------------------------------------------
>
> Key: BIGTOP-710
> URL: https://issues.apache.org/jira/browse/BIGTOP-710
> Project: Bigtop
> Issue Type: Improvement
> Components: Deployment
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.5.0
>
>
> It would be very nice to start using our puppet code as a basis for a higher level orchestration framework capable of driving a true distributed multi-node deployment of Bigtop components. Puppet code is great at stamping out a predictable single node deployments, but we have to extend it with a DSL aimed at solving higher level orchestration.
> This JIRA is meant to be a central information radiator for collecting ideas on how to approach this (what DSL to use, how to structure communication between agents and fact sharing, etc).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-710) Create a higher level orchestration
deployment framework
Posted by "Bruno Mahé (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461525#comment-13461525 ]
Bruno Mahé commented on BIGTOP-710:
-----------------------------------
Cos> I am not sure to follow. Where was it stated that this ticket is not somewhat different from BIGTOP-635? I stated they were related and that discussions about BIGTOP-635 brought up other discussions about such framework, but nothing further than that. My point being this ticket could be used as one of the implementation available for BIGTOP-635 and there may be a need for an api for the uses cases of this ticket.
> Create a higher level orchestration deployment framework
> --------------------------------------------------------
>
> Key: BIGTOP-710
> URL: https://issues.apache.org/jira/browse/BIGTOP-710
> Project: Bigtop
> Issue Type: Improvement
> Components: Deployment
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Labels: orchestration
> Fix For: 0.5.0
>
>
> It would be very nice to start using our puppet code as a basis for a higher level orchestration framework capable of driving a true distributed multi-node deployment of Bigtop components. Puppet code is great at stamping out a predictable single node deployments, but we have to extend it with a DSL aimed at solving higher level orchestration.
> This JIRA is meant to be a central information radiator for collecting ideas on how to approach this (what DSL to use, how to structure communication between agents and fact sharing, etc).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-710) Create a higher level orchestration
deployment framework
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461483#comment-13461483 ]
Konstantin Boudnik commented on BIGTOP-710:
-------------------------------------------
Bruno, good points. We certainly need to have a clear understanding of the goal, before we embark on any journey.
I think this ticket is somewhat different. To my, the orchestration is about 'management of a bunch of nodes (i.e. cluster) via deployment, control and monitoring of BigTop produced artifacts'
This is quite different from BIGTOP-635 and alike, IMO.
> Create a higher level orchestration deployment framework
> --------------------------------------------------------
>
> Key: BIGTOP-710
> URL: https://issues.apache.org/jira/browse/BIGTOP-710
> Project: Bigtop
> Issue Type: Improvement
> Components: Deployment
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Labels: orchestration
> Fix For: 0.5.0
>
>
> It would be very nice to start using our puppet code as a basis for a higher level orchestration framework capable of driving a true distributed multi-node deployment of Bigtop components. Puppet code is great at stamping out a predictable single node deployments, but we have to extend it with a DSL aimed at solving higher level orchestration.
> This JIRA is meant to be a central information radiator for collecting ideas on how to approach this (what DSL to use, how to structure communication between agents and fact sharing, etc).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (BIGTOP-710) Create a higher level
orchestration deployment framework
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460226#comment-13460226 ]
Konstantin Boudnik edited comment on BIGTOP-710 at 9/21/12 4:30 PM:
--------------------------------------------------------------------
@Johnny: I don't think this is about system tests, although a part of any good orchestration framework would be a set of behavioral components tests that can provide a certain guarantees that brought up system/cluster is up to the specs, and working.
was (Author: cos):
@Johnny: I don't think this is about system tests, although a part of any good orchestration framework would be a good set of behavioral components tests that can provide a certain guarantees that brought up system/cluster is up to the specs, and working.
> Create a higher level orchestration deployment framework
> --------------------------------------------------------
>
> Key: BIGTOP-710
> URL: https://issues.apache.org/jira/browse/BIGTOP-710
> Project: Bigtop
> Issue Type: Improvement
> Components: Deployment
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Labels: orchestration
> Fix For: 0.5.0
>
>
> It would be very nice to start using our puppet code as a basis for a higher level orchestration framework capable of driving a true distributed multi-node deployment of Bigtop components. Puppet code is great at stamping out a predictable single node deployments, but we have to extend it with a DSL aimed at solving higher level orchestration.
> This JIRA is meant to be a central information radiator for collecting ideas on how to approach this (what DSL to use, how to structure communication between agents and fact sharing, etc).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-710) Create a higher level orchestration
deployment framework
Posted by "Anatoli Fomenko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460578#comment-13460578 ]
Anatoli Fomenko commented on BIGTOP-710:
----------------------------------------
I'm wondering if the frameworks mentioned here would be suitable as a foundation for building a robust open cluster manager?
> Create a higher level orchestration deployment framework
> --------------------------------------------------------
>
> Key: BIGTOP-710
> URL: https://issues.apache.org/jira/browse/BIGTOP-710
> Project: Bigtop
> Issue Type: Improvement
> Components: Deployment
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Labels: orchestration
> Fix For: 0.5.0
>
>
> It would be very nice to start using our puppet code as a basis for a higher level orchestration framework capable of driving a true distributed multi-node deployment of Bigtop components. Puppet code is great at stamping out a predictable single node deployments, but we have to extend it with a DSL aimed at solving higher level orchestration.
> This JIRA is meant to be a central information radiator for collecting ideas on how to approach this (what DSL to use, how to structure communication between agents and fact sharing, etc).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-710) Create a higher level orchestration
deployment framework
Posted by "Bruno Mahé (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460214#comment-13460214 ]
Bruno Mahé commented on BIGTOP-710:
-----------------------------------
Has anyone taken a look at mcollective? I haven't had time to try yet but from its front page http://puppetlabs.com/mcollective/introduction/ it seems to do exactly the intent of this ticket:
{quote}
The Marionette Collective AKA MCollective is a framework to build server orchestration or parallel job execution systems.
{quote}
It seems also to not be always tied to puppet:
{quote}
Break free from ever more complex naming conventions for hostnames as a means of identity. Use a very rich set of meta data provided by each machine to address them. Meta data comes from Puppet, Chef, Facter, Ohai or plugins you provide yourself.
{quote}
> Create a higher level orchestration deployment framework
> --------------------------------------------------------
>
> Key: BIGTOP-710
> URL: https://issues.apache.org/jira/browse/BIGTOP-710
> Project: Bigtop
> Issue Type: Improvement
> Components: Deployment
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.5.0
>
>
> It would be very nice to start using our puppet code as a basis for a higher level orchestration framework capable of driving a true distributed multi-node deployment of Bigtop components. Puppet code is great at stamping out a predictable single node deployments, but we have to extend it with a DSL aimed at solving higher level orchestration.
> This JIRA is meant to be a central information radiator for collecting ideas on how to approach this (what DSL to use, how to structure communication between agents and fact sharing, etc).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-710) Create a higher level orchestration
deployment framework
Posted by "Bruno Mahé (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461010#comment-13461010 ]
Bruno Mahé commented on BIGTOP-710:
-----------------------------------
Anatoli> It could, but I would not go there.
Building a robust open cluster manager has a much larger scope than building something solving Apache Bigtop issues of today. Before trying to solve everyone's problem, I would rather try to solve Apache Bigtop's first. We can always expand later on.
Which brings me to the point that this ticket try to state a solution to a problem not yet defined.
We all know and talked about wanting some sort of orchestration layer to help us managing our VMs and test nodes (see BIGTOP-635 for instance) but we probably all have something different in mind. This may create some confusion later when implementations starts to appears but not matching everyone's expectations.
So maybe we should spend some time first defining the use cases and requirements?
> Create a higher level orchestration deployment framework
> --------------------------------------------------------
>
> Key: BIGTOP-710
> URL: https://issues.apache.org/jira/browse/BIGTOP-710
> Project: Bigtop
> Issue Type: Improvement
> Components: Deployment
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Labels: orchestration
> Fix For: 0.5.0
>
>
> It would be very nice to start using our puppet code as a basis for a higher level orchestration framework capable of driving a true distributed multi-node deployment of Bigtop components. Puppet code is great at stamping out a predictable single node deployments, but we have to extend it with a DSL aimed at solving higher level orchestration.
> This JIRA is meant to be a central information radiator for collecting ideas on how to approach this (what DSL to use, how to structure communication between agents and fact sharing, etc).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (BIGTOP-710) Create a higher level
orchestration deployment framework
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460220#comment-13460220 ]
Konstantin Boudnik edited comment on BIGTOP-710 at 9/21/12 4:19 PM:
--------------------------------------------------------------------
I almost feel Glu ;)
And MCollectives too ;)
was (Author: cos):
I almost feel Glu ;)
> Create a higher level orchestration deployment framework
> --------------------------------------------------------
>
> Key: BIGTOP-710
> URL: https://issues.apache.org/jira/browse/BIGTOP-710
> Project: Bigtop
> Issue Type: Improvement
> Components: Deployment
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.5.0
>
>
> It would be very nice to start using our puppet code as a basis for a higher level orchestration framework capable of driving a true distributed multi-node deployment of Bigtop components. Puppet code is great at stamping out a predictable single node deployments, but we have to extend it with a DSL aimed at solving higher level orchestration.
> This JIRA is meant to be a central information radiator for collecting ideas on how to approach this (what DSL to use, how to structure communication between agents and fact sharing, etc).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-710) Create a higher level orchestration
deployment framework
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461998#comment-13461998 ]
Konstantin Boudnik commented on BIGTOP-710:
-------------------------------------------
it has not been. I simply mentioned it to re-enforce my view of the orchestration and this it is different from what BIGTOP-635 is about. I agree that BIGTOP-710 would be a solid foundation of the other one. Sorry for any confusion ;)
> Create a higher level orchestration deployment framework
> --------------------------------------------------------
>
> Key: BIGTOP-710
> URL: https://issues.apache.org/jira/browse/BIGTOP-710
> Project: Bigtop
> Issue Type: Improvement
> Components: Deployment
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Labels: orchestration
> Fix For: 0.5.0
>
>
> It would be very nice to start using our puppet code as a basis for a higher level orchestration framework capable of driving a true distributed multi-node deployment of Bigtop components. Puppet code is great at stamping out a predictable single node deployments, but we have to extend it with a DSL aimed at solving higher level orchestration.
> This JIRA is meant to be a central information radiator for collecting ideas on how to approach this (what DSL to use, how to structure communication between agents and fact sharing, etc).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-710) Create a higher level orchestration
deployment framework
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460226#comment-13460226 ]
Konstantin Boudnik commented on BIGTOP-710:
-------------------------------------------
@Johnny: I don't think this is about system tests, although a part of any good orchestration framework would be a good set of behavioral components tests that can provide a certain guarantees that brought up system/cluster is up to the specs, and working.
> Create a higher level orchestration deployment framework
> --------------------------------------------------------
>
> Key: BIGTOP-710
> URL: https://issues.apache.org/jira/browse/BIGTOP-710
> Project: Bigtop
> Issue Type: Improvement
> Components: Deployment
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Labels: orchestration
> Fix For: 0.5.0
>
>
> It would be very nice to start using our puppet code as a basis for a higher level orchestration framework capable of driving a true distributed multi-node deployment of Bigtop components. Puppet code is great at stamping out a predictable single node deployments, but we have to extend it with a DSL aimed at solving higher level orchestration.
> This JIRA is meant to be a central information radiator for collecting ideas on how to approach this (what DSL to use, how to structure communication between agents and fact sharing, etc).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira