[DISCUSS] New subproject Aries Containers for Container Management

[David Bosschaert <da...@gmail.com>]

Hi all,

Over the past years I have been working on an API that provides a layer of
abstraction for container management. What it basically allows you to do is
create a program that can run docker containers (or potentially other
containers) regardless of the underlying infrastructure.

A little history of the project to provide some context. I worked (and am
still working) on projects that require applications to be run as docker
containers. Initially I started this as a pure docker project, but later it
was moved to use Amazon ECS, a docker container environment. Later it
migrated to use Marathon and later yet again it migrated to another system
for managing docker containers.
Lucky enough I had an API abstraction in place which allowed me to take the
existing application and I just had to write a new driver for whatever
today's platform of deployment was going to be.

The ideas and insights I gained during this time led me to submit OSGi RPC
179 which has since been accepted by the OSGi community. While this is
currently just an RFP (no detailed technical design yet) the idea could be
there that different communities can implement this API to focus on the
backend that they care about. This has two benefits:
1. users can switch backend by just taking the implementation that supports
it
2. implementations don't need to support every backend that exists in the
world. They rather focus on a small number that they care about, which
keeps this maintenance of this manageable.

While I have quite a lot of code for this personally think that it would be
better to start from scratch here. So, I would propose to start a new
subproject in Apache Aries, for example named 'Aries Containers'. There we
could implement the start of what could become input to the OSGi API and
maybe a small number of implementations, for example a Marathon one and
maybe a bare Docker one. We might also consider providing a Kuberbetes impl
if someone with knowledge of that (I have not used that much) joins the fun.

Thoughts anyone? If people are happy with this idea my plan is to start
adding some API and impl soon to provide a starting point for such a
project.

Best regards,

David

[1]
https://github.com/osgi/design/blob/master/rfps/rfp-0179-ComputeManagementService.pdf

Re: [DISCUSS] New subproject Aries Containers for Container Management

[David Bosschaert <da...@gmail.com>]

<re-adding dev@aries>

Thanks Christian, I'll give that a try.

Cheers,

David

On 23 May 2017 at 13:37, Christian Schneider <ch...@die-schneider.net>
wrote:

> You can request a repo using this form:
>
> https://reporeq.apache.org/
>
> Christian
>
> On 23.05.2017 14:34, David Bosschaert wrote:
>
>> Git sounds good to me. Does anyone have a pointer re how to get the repo
>> created?
>>
>> Cheers,
>>
>> David
>>
>>
> --
> Christian Schneider
> http://www.liquid-reality.de
>
> Open Source Architect
> http://www.talend.com
>
>

Re: [DISCUSS] New subproject Aries Containers for Container Management

[David Bosschaert <da...@gmail.com>]

Git sounds good to me. Does anyone have a pointer re how to get the repo
created?

Cheers,

David

On 23 May 2017 at 11:43, Dominik Przybysz <al...@gmail.com> wrote:

> Hi,
> new git repo is better in my opinion.
>
> 2017-05-23 11:43 GMT+02:00 David Bosschaert <da...@gmail.com>:
>
> > Not a lot of discussion yet - one thing to add is that I did a
> presentation
> > together with Carsten Ziegeler about this topic some time ago:
> > https://www.slideshare.net/mfrancis/dockerizing-apps-
> > for-the-deployment-platform-of-the-month-with-osgi-david-
> > bosschaert-carsten-ziegeler
> > where some of the ideas are covered.
> >
> > If nobody complains I'll start a little subproject in the Aries SVN where
> > the ideas can be worked out further. Or should I create an Aries Git repo
> > for this?
> >
> > Cheers,
> >
> > David
> >
> > On 16 May 2017 at 08:46, David Bosschaert <da...@gmail.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > Over the past years I have been working on an API that provides a layer
> > of
> > > abstraction for container management. What it basically allows you to
> do
> > is
> > > create a program that can run docker containers (or potentially other
> > > containers) regardless of the underlying infrastructure.
> > >
> > > A little history of the project to provide some context. I worked (and
> am
> > > still working) on projects that require applications to be run as
> docker
> > > containers. Initially I started this as a pure docker project, but
> later
> > it
> > > was moved to use Amazon ECS, a docker container environment. Later it
> > > migrated to use Marathon and later yet again it migrated to another
> > system
> > > for managing docker containers.
> > > Lucky enough I had an API abstraction in place which allowed me to take
> > > the existing application and I just had to write a new driver for
> > whatever
> > > today's platform of deployment was going to be.
> > >
> > > The ideas and insights I gained during this time led me to submit OSGi
> > RPC
> > > 179 which has since been accepted by the OSGi community. While this is
> > > currently just an RFP (no detailed technical design yet) the idea could
> > be
> > > there that different communities can implement this API to focus on the
> > > backend that they care about. This has two benefits:
> > > 1. users can switch backend by just taking the implementation that
> > > supports it
> > > 2. implementations don't need to support every backend that exists in
> the
> > > world. They rather focus on a small number that they care about, which
> > > keeps this maintenance of this manageable.
> > >
> > > While I have quite a lot of code for this personally think that it
> would
> > > be better to start from scratch here. So, I would propose to start a
> new
> > > subproject in Apache Aries, for example named 'Aries Containers'. There
> > we
> > > could implement the start of what could become input to the OSGi API
> and
> > > maybe a small number of implementations, for example a Marathon one and
> > > maybe a bare Docker one. We might also consider providing a Kuberbetes
> > impl
> > > if someone with knowledge of that (I have not used that much) joins the
> > fun.
> > >
> > > Thoughts anyone? If people are happy with this idea my plan is to start
> > > adding some API and impl soon to provide a starting point for such a
> > > project.
> > >
> > > Best regards,
> > >
> > > David
> > >
> > > [1] https://github.com/osgi/design/blob/master/rfps/rfp-0179
> > > -ComputeManagementService.pdf
> > >
> >
>
>
>
> --
> Pozdrawiam / Regards,
> Dominik Przybysz
>

Re: [DISCUSS] New subproject Aries Containers for Container Management

[Dominik Przybysz <al...@gmail.com>]

Hi,
new git repo is better in my opinion.

2017-05-23 11:43 GMT+02:00 David Bosschaert <da...@gmail.com>:

> Not a lot of discussion yet - one thing to add is that I did a presentation
> together with Carsten Ziegeler about this topic some time ago:
> https://www.slideshare.net/mfrancis/dockerizing-apps-
> for-the-deployment-platform-of-the-month-with-osgi-david-
> bosschaert-carsten-ziegeler
> where some of the ideas are covered.
>
> If nobody complains I'll start a little subproject in the Aries SVN where
> the ideas can be worked out further. Or should I create an Aries Git repo
> for this?
>
> Cheers,
>
> David
>
> On 16 May 2017 at 08:46, David Bosschaert <da...@gmail.com>
> wrote:
>
> > Hi all,
> >
> > Over the past years I have been working on an API that provides a layer
> of
> > abstraction for container management. What it basically allows you to do
> is
> > create a program that can run docker containers (or potentially other
> > containers) regardless of the underlying infrastructure.
> >
> > A little history of the project to provide some context. I worked (and am
> > still working) on projects that require applications to be run as docker
> > containers. Initially I started this as a pure docker project, but later
> it
> > was moved to use Amazon ECS, a docker container environment. Later it
> > migrated to use Marathon and later yet again it migrated to another
> system
> > for managing docker containers.
> > Lucky enough I had an API abstraction in place which allowed me to take
> > the existing application and I just had to write a new driver for
> whatever
> > today's platform of deployment was going to be.
> >
> > The ideas and insights I gained during this time led me to submit OSGi
> RPC
> > 179 which has since been accepted by the OSGi community. While this is
> > currently just an RFP (no detailed technical design yet) the idea could
> be
> > there that different communities can implement this API to focus on the
> > backend that they care about. This has two benefits:
> > 1. users can switch backend by just taking the implementation that
> > supports it
> > 2. implementations don't need to support every backend that exists in the
> > world. They rather focus on a small number that they care about, which
> > keeps this maintenance of this manageable.
> >
> > While I have quite a lot of code for this personally think that it would
> > be better to start from scratch here. So, I would propose to start a new
> > subproject in Apache Aries, for example named 'Aries Containers'. There
> we
> > could implement the start of what could become input to the OSGi API and
> > maybe a small number of implementations, for example a Marathon one and
> > maybe a bare Docker one. We might also consider providing a Kuberbetes
> impl
> > if someone with knowledge of that (I have not used that much) joins the
> fun.
> >
> > Thoughts anyone? If people are happy with this idea my plan is to start
> > adding some API and impl soon to provide a starting point for such a
> > project.
> >
> > Best regards,
> >
> > David
> >
> > [1] https://github.com/osgi/design/blob/master/rfps/rfp-0179
> > -ComputeManagementService.pdf
> >
>



-- 
Pozdrawiam / Regards,
Dominik Przybysz

Re: [DISCUSS] New subproject Aries Containers for Container Management

[David Bosschaert <da...@gmail.com>]

Not a lot of discussion yet - one thing to add is that I did a presentation
together with Carsten Ziegeler about this topic some time ago:
https://www.slideshare.net/mfrancis/dockerizing-apps-
for-the-deployment-platform-of-the-month-with-osgi-david-
bosschaert-carsten-ziegeler
where some of the ideas are covered.

If nobody complains I'll start a little subproject in the Aries SVN where
the ideas can be worked out further. Or should I create an Aries Git repo
for this?

Cheers,

David

On 16 May 2017 at 08:46, David Bosschaert <da...@gmail.com>
wrote:

> Hi all,
>
> Over the past years I have been working on an API that provides a layer of
> abstraction for container management. What it basically allows you to do is
> create a program that can run docker containers (or potentially other
> containers) regardless of the underlying infrastructure.
>
> A little history of the project to provide some context. I worked (and am
> still working) on projects that require applications to be run as docker
> containers. Initially I started this as a pure docker project, but later it
> was moved to use Amazon ECS, a docker container environment. Later it
> migrated to use Marathon and later yet again it migrated to another system
> for managing docker containers.
> Lucky enough I had an API abstraction in place which allowed me to take
> the existing application and I just had to write a new driver for whatever
> today's platform of deployment was going to be.
>
> The ideas and insights I gained during this time led me to submit OSGi RPC
> 179 which has since been accepted by the OSGi community. While this is
> currently just an RFP (no detailed technical design yet) the idea could be
> there that different communities can implement this API to focus on the
> backend that they care about. This has two benefits:
> 1. users can switch backend by just taking the implementation that
> supports it
> 2. implementations don't need to support every backend that exists in the
> world. They rather focus on a small number that they care about, which
> keeps this maintenance of this manageable.
>
> While I have quite a lot of code for this personally think that it would
> be better to start from scratch here. So, I would propose to start a new
> subproject in Apache Aries, for example named 'Aries Containers'. There we
> could implement the start of what could become input to the OSGi API and
> maybe a small number of implementations, for example a Marathon one and
> maybe a bare Docker one. We might also consider providing a Kuberbetes impl
> if someone with knowledge of that (I have not used that much) joins the fun.
>
> Thoughts anyone? If people are happy with this idea my plan is to start
> adding some API and impl soon to provide a starting point for such a
> project.
>
> Best regards,
>
> David
>
> [1] https://github.com/osgi/design/blob/master/rfps/rfp-0179
> -ComputeManagementService.pdf
>

Re: [DISCUSS] New subproject Aries Containers for Container Management

[David Bosschaert <da...@gmail.com>]

Hi all,

I started a very small implementation of this at
https://git-wip-us.apache.org/repos/asf/aries-containers.git - it currently
provides a tiny API that still needs to grow a little and the start of
implementations for local Docker use and for use with Marathon.

I also added some examples that are documented here:
http://aries.apache.org/modules/containers.html

Over the next little while I'm intending to build out the API a little,
although I think it would be nice if the API can stay relatively small. I'm
also planning to add additional unit tests.

It would also be nice if others have experience with other container
management systems, such as Kubernetes, to provide a few more bindings...

Enjoy,

David


On 24 May 2017 at 14:34, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:

> Hi David,
>
> Very excited by this.
>
> By the way, last week at ApacheCon, I did a talk: "Large and scalable
> Apache Karaf cluster with Karaf Cellar and Mesos/Marathon". I showed how to
> use Marathon (via DC/OS) with Cellar to scala. I started to implement a
> marathon scheduler in Cellar.
>
> So, it sounds pretty close.
>
> Aries Containers is a good name IMHO. We can start this as an aries
> subproject, maybe isolated on a dedicated git.
>
> Thanks !
> Regards
> JB
>
>
> On 05/16/2017 09:46 AM, David Bosschaert wrote:
>
>> Hi all,
>>
>> Over the past years I have been working on an API that provides a layer of
>> abstraction for container management. What it basically allows you to do
>> is
>> create a program that can run docker containers (or potentially other
>> containers) regardless of the underlying infrastructure.
>>
>> A little history of the project to provide some context. I worked (and am
>> still working) on projects that require applications to be run as docker
>> containers. Initially I started this as a pure docker project, but later
>> it
>> was moved to use Amazon ECS, a docker container environment. Later it
>> migrated to use Marathon and later yet again it migrated to another system
>> for managing docker containers.
>> Lucky enough I had an API abstraction in place which allowed me to take
>> the
>> existing application and I just had to write a new driver for whatever
>> today's platform of deployment was going to be.
>>
>> The ideas and insights I gained during this time led me to submit OSGi RPC
>> 179 which has since been accepted by the OSGi community. While this is
>> currently just an RFP (no detailed technical design yet) the idea could be
>> there that different communities can implement this API to focus on the
>> backend that they care about. This has two benefits:
>> 1. users can switch backend by just taking the implementation that
>> supports
>> it
>> 2. implementations don't need to support every backend that exists in the
>> world. They rather focus on a small number that they care about, which
>> keeps this maintenance of this manageable.
>>
>> While I have quite a lot of code for this personally think that it would
>> be
>> better to start from scratch here. So, I would propose to start a new
>> subproject in Apache Aries, for example named 'Aries Containers'. There we
>> could implement the start of what could become input to the OSGi API and
>> maybe a small number of implementations, for example a Marathon one and
>> maybe a bare Docker one. We might also consider providing a Kuberbetes
>> impl
>> if someone with knowledge of that (I have not used that much) joins the
>> fun.
>>
>> Thoughts anyone? If people are happy with this idea my plan is to start
>> adding some API and impl soon to provide a starting point for such a
>> project.
>>
>> Best regards,
>>
>> David
>>
>> [1]
>> https://github.com/osgi/design/blob/master/rfps/rfp-0179-
>> ComputeManagementService.pdf
>>
>>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Re: [DISCUSS] New subproject Aries Containers for Container Management

[Jean-Baptiste Onofré <jb...@nanthrax.net>]

Hi David,

Very excited by this.

By the way, last week at ApacheCon, I did a talk: "Large and scalable Apache 
Karaf cluster with Karaf Cellar and Mesos/Marathon". I showed how to use 
Marathon (via DC/OS) with Cellar to scala. I started to implement a marathon 
scheduler in Cellar.

So, it sounds pretty close.

Aries Containers is a good name IMHO. We can start this as an aries subproject, 
maybe isolated on a dedicated git.

Thanks !
Regards
JB

On 05/16/2017 09:46 AM, David Bosschaert wrote:
> Hi all,
> 
> Over the past years I have been working on an API that provides a layer of
> abstraction for container management. What it basically allows you to do is
> create a program that can run docker containers (or potentially other
> containers) regardless of the underlying infrastructure.
> 
> A little history of the project to provide some context. I worked (and am
> still working) on projects that require applications to be run as docker
> containers. Initially I started this as a pure docker project, but later it
> was moved to use Amazon ECS, a docker container environment. Later it
> migrated to use Marathon and later yet again it migrated to another system
> for managing docker containers.
> Lucky enough I had an API abstraction in place which allowed me to take the
> existing application and I just had to write a new driver for whatever
> today's platform of deployment was going to be.
> 
> The ideas and insights I gained during this time led me to submit OSGi RPC
> 179 which has since been accepted by the OSGi community. While this is
> currently just an RFP (no detailed technical design yet) the idea could be
> there that different communities can implement this API to focus on the
> backend that they care about. This has two benefits:
> 1. users can switch backend by just taking the implementation that supports
> it
> 2. implementations don't need to support every backend that exists in the
> world. They rather focus on a small number that they care about, which
> keeps this maintenance of this manageable.
> 
> While I have quite a lot of code for this personally think that it would be
> better to start from scratch here. So, I would propose to start a new
> subproject in Apache Aries, for example named 'Aries Containers'. There we
> could implement the start of what could become input to the OSGi API and
> maybe a small number of implementations, for example a Marathon one and
> maybe a bare Docker one. We might also consider providing a Kuberbetes impl
> if someone with knowledge of that (I have not used that much) joins the fun.
> 
> Thoughts anyone? If people are happy with this idea my plan is to start
> adding some API and impl soon to provide a starting point for such a
> project.
> 
> Best regards,
> 
> David
> 
> [1]
> https://github.com/osgi/design/blob/master/rfps/rfp-0179-ComputeManagementService.pdf
> 

-- 
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com