[DISCUSS] Docker Integration (Regular ActiveMQ & ActiveMQ Artemis?)

["John D. Ament" <jo...@apache.org>]

All,

I wanted to bring up the topic of how to do docker integration.  There's
some discussions I saw pop up on both the regular ActiveMQ distribution as
well as the Artemis project.  This started from a potential donation to
Artemis to add a dockerfile (its apparently pretty popular on dockerhub -
https://hub.docker.com/r/vromero/activemq-artemis/ )

I've gotten some concrete information from infra.  There's two ways we do
docker integration at the ASF - automatic builds to dockerhub or manual
builds to bintray.  The problem with the dockerhub approach is that its
based on git tags or pushes to branches.  We can setup some rules around
what does or does not get pushed, but ultimately its all static contents.
Its also manual only.  In talking with infra, there's some stuff that can
be done to retroactively add builds, or replace contents, but its nothing
too robust.  The bintray approach seems to be the stronger option though.
Its based on bintray's APIs, meaning we can push during a release the
artifacts up to bintray.  I believe there is syncing to dockerhub as well,
so by pushing (as a part of a release) to bintray it could sync the docker
container up to the central hub.  It does give the PMC better control over
what goes where, and would allow ActiveMQ to self-manage the release
process of the containers.

One interesting thing.  The main way to integrate into a build is to use
the docker-maven -plugin (since we're already using maven),
https://github.com/spotify/docker-maven-plugin .   This would allow pushing
based on the dockerfile, and combined with our normal pom files.  It also
means it could be fully automated as a part of the maven release process
which is pretty nice.

I will also point out that there are some concerns over a handful of the
integration strategies out there, so unfortunately the data is a bit jagged
so far.  We don't want to have a docker container that does a build as a
part of its spin up.  We also don't want to have pre-release software out
on docker containers, similar to the usage of maven central (for releases
only) but having a nexus based repository for snapshots for devs who want
to try the release early on.

So I'm putting all this out there (and didn't think this was going to end
up being such a long email!).  I want to understand from the broader group
what type of integration strategy should be used.

John

Re: [DISCUSS] Docker Integration (Regular ActiveMQ & ActiveMQ Artemis?)

[Victor <vi...@gmail.com>]

Hi John et all,

You mention the following the solutions :

   1. Maven->spotify plugin->bintray
   2. Dockerhub automatic from same repo


Regarding 1, having packages _just_ in bintray (or quay.io, nexus, etc)
defeats the purpose of having users for the software.
Dockerhub is by far the most used repository, a good portion of the users
will search dockerhub and if something doesn't exist there they will desist
(although I do understand others could follow documentation, they won't be
the majority).
Also It would be specially confusing if some of the apache docker images
are in dockerhub and others are in bintray.

Around 2, I havent heard of any bintray <-> dockerhub integration off-the-shelf
solution out there.
If Apache has one already that could be a perfectly great solution.
It would be mostly maintenance from the release perspective and although
the automatic builds are preferred in dockerhub, manual builds are still
perfectly fine.

The third solution would be the dockerhub automatic approach in the same
repository of activemq, but it would have two big cons:
Pros:
- Just one repo
Cons:
- right after release and publication it would require manual maintenance
of the dockerfile
- on release it would require some dockerhub configuration modifications.
The dockerhub maintenance will likely become a bottleneck as not everybody
can have access to the apache dockerhub account.
It would be a particularly awkward, in the case of artemis, where there is
apparently just the master branch.
- Very difficult to release security updates of older versions

And last but not least, the natural dockerhub approach, a different git
repostiory holding branches for each docker version released (this is what
I use today for https://hub.docker.com/r/vromero/activemq-artemis/):
Pros:
- What everybody is doing and expecting to find. We can expect community
contributions.
- Enables us to release security updates very quick and easy.
Cons:
- Still requires some manual configuration of dockerhub
- Requires some work after releases, but I'm keen on doing it.

I guess the other more sophisticated build and repository system: quay.io,
with regular expression based branch matchers and security scanners is out
of the game. And regardless It would require dockerhub integration too.

I personally lean towards the natural dockerhub approach. I would expect
dockerhub to enable quay's regular expression features in the near future
and there is great value in enabling the community to contribute easily.

If that doesn't work, the maven->spotify->bintray->integration->dockerhub
is perfectly ok, but I don't see what stop us to just
maven->spotify->dockerhub.

Just my two cents.




2016-08-31 18:17 GMT-07:00 John D. Ament <jo...@apache.org>:

> All,
>
> I wanted to bring up the topic of how to do docker integration.  There's
> some discussions I saw pop up on both the regular ActiveMQ distribution as
> well as the Artemis project.  This started from a potential donation to
> Artemis to add a dockerfile (its apparently pretty popular on dockerhub -
> https://hub.docker.com/r/vromero/activemq-artemis/ )
>
> I've gotten some concrete information from infra.  There's two ways we do
> docker integration at the ASF - automatic builds to dockerhub or manual
> builds to bintray.  The problem with the dockerhub approach is that its
> based on git tags or pushes to branches.  We can setup some rules around
> what does or does not get pushed, but ultimately its all static contents.
> Its also manual only.  In talking with infra, there's some stuff that can
> be done to retroactively add builds, or replace contents, but its nothing
> too robust.  The bintray approach seems to be the stronger option though.
> Its based on bintray's APIs, meaning we can push during a release the
> artifacts up to bintray.  I believe there is syncing to dockerhub as well,
> so by pushing (as a part of a release) to bintray it could sync the docker
> container up to the central hub.  It does give the PMC better control over
> what goes where, and would allow ActiveMQ to self-manage the release
> process of the containers.
>
> One interesting thing.  The main way to integrate into a build is to use
> the docker-maven -plugin (since we're already using maven),
> https://github.com/spotify/docker-maven-plugin .   This would allow
> pushing
> based on the dockerfile, and combined with our normal pom files.  It also
> means it could be fully automated as a part of the maven release process
> which is pretty nice.
>
> I will also point out that there are some concerns over a handful of the
> integration strategies out there, so unfortunately the data is a bit jagged
> so far.  We don't want to have a docker container that does a build as a
> part of its spin up.  We also don't want to have pre-release software out
> on docker containers, similar to the usage of maven central (for releases
> only) but having a nexus based repository for snapshots for devs who want
> to try the release early on.
>
> So I'm putting all this out there (and didn't think this was going to end
> up being such a long email!).  I want to understand from the broader group
> what type of integration strategy should be used.
>
> John
>