You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Irina Boverman <ib...@redhat.com> on 2017/06/13 14:13:15 UTC
Proposal to create Docker images for Qpid components.
Hi everyone,
I would like to propose creating Docker images for Qpid components hosted
in Docker Hub, updated upon component release and maintained by the
project, and I would like to contribute to doing this.
Availability of Qpid images will make it easier to consume/deploy Qpid
components and promote Qpid visibility.
We can maintain docker scripts creating these images from the base OS
images and using Qpid installation methods consistent with the OS
distribution. A possible naming convention might be qpid/<component>/<OS>.
I registered the 'qpid' user on DockerHub to use if this seems reasonable.
For example, we could create qpid/dispatch/<OS> image, qpid/<broker>/<OS>
image, qpid/<client(s)>/<OS> image, etc. Initially I would look to support
Fedora/CentOS latest images and Qpid components as RPMs for them, then aim
to expand OS coverage for debian/Ubuntu/etc in the future.
The goal would be to update Qpid images within a few days upon component
release (either directly or indirectly using yum/dnf from public
repositories). We could ask the Docker team to grant Qpid "official" status
when images have been stabilized.
--
Regards, Irina.
Re: Proposal to create Docker images for Qpid components.
Posted by Irina Boverman <ib...@redhat.com>.
Thank you for excellent remarks. I will take a look at your work and what
can be done to minimise images, although I must admit it is easier to layer
on top of official images produced by the distros.
On Tue, Jun 13, 2017 at 11:36 AM, Fraser Adams <
fraser.adams@blueyonder.co.uk> wrote:
> Hi Irina,
>
> I think that this is an excellent idea and I totally agree with the
> sentiment "Availability of Qpid images will make it easier to
> consume/deploy Qpid components and promote Qpid visibility." I think that
> in 2017 it's really important for the project to be providing official
> maintained images as otherwise we'll continue to see a fragmented
> miscellany of images of unknown provenance growing up out of necessity. To
> be fair that seems to be a general trend on Docker Hub for many projects,
> but I personally tend to find it a bit frustrating and would prefer a bit
> more control over provenance and versioning for things used on production
> systems.
>
> I'd make a few comments though.
>
> 1. It's probably better to propose this on the users list as that tends to
> have more traffic than the dev list. If you choose to repost on the users
> list feel free to include this response (if it's useful).
>
> 2. I do wonder about the comment "We can maintain docker scripts creating
> these images from the base OS images and using Qpid installation methods
> consistent with the OS distribution" and "I would look to support
> Fedora/CentOS latest images" and "expand OS coverage for debian/Ubuntu/etc".
>
> I can see why this might be the /obvious/ approach, but IMHO I think that
> it's a somewhat poor anti-pattern for containerisation that is propagated
> way too often on Docker Hub and tends to result in large bloaty images with
> larger potential attack surfaces than necessary and in addition reinforces
> a model that containers are somehow "lightweight VMs" rather than
> containerised appliances. I very much prefer a micro-containerised approach
> using an approach similar to that descrived here:
> http://blog.oddbit.com/2015/02/05/creating-minimal-docker-images/ though
> acknowledging that this approach adds some complexity to the build process
> and requires a little care.
>
> Ultimately reducing image size from the 400-500MB range to something like
> 55MB for qpidd feels like a good thing for a number of reasons, the
> following article contains some interesting musings on the subject
> https://opensolitude.com/2015/05/13/docker-images-should-be-small.html.
>
> It's a bit more complex for Python and Java based components, but I think
> that basing components on lighter weight Alpine Linux based Java or Python
> runtimes is probably the best way to go, I note that Docker Hub is (slowly)
> moving towards Alpine and http://www.iron.io/ have built some uber tiny
> Docker images for a wide range of languages see
> https://github.com/iron-io/dockers and https://hub.docker.com/u/iron/
>
> 3. One question to ask is which Qpid versions would be supported by the
> build system. Obviously one advantage of containerisation is the ability to
> easily use multiple versions of an application in order to support
> regression testing and potentially legacy capabilities in an operational
> environment, but of course there is a price to be paid as older versions
> run into difficulties with newer compilers, boost versions etc.
>
>
> I had a bit of a go at creating a build system for qpidd, my intention has
> been to do the same for other components, but I've been stuck for time, see
> https://github.com/fadams/docker-qpid though the qpid-cpp link is the
> only one with any useful content.
>
> The approach that I took was to create a Dockerised build system based on
> Debian so that the only dependency to actually build the component is a
> working Docker install and an Internet connection, the build system
> installs the necessary dev packages then installs the requested qpid-cpp
> source, runs the Qpid build then it exports the relevant shared libraries
> etc. and dynamic linker in order that a separate Dockerfile can use these
> to create a small image containing only what is necessary for qpidd to
> execute.
>
> I created two alternative build systems, one based on Debian wheezy the
> other based on jessie, the former was somewhat simpler as wheezy has
> somewhat older gcc and boost packages so less "wrangling" was necessary to
> build really old Qpid versions, with jessie I had to build gcc and boost
> from source. If you only intent to build recent Qpid versions much of that
> complexity is unnecessary as recent Qpid versions compile with recent gcc
> and boost versions, but I'm a little bit of a glutton for punishment and it
> seemed a bit more of a challenge to try and create something that could
> build every qpidd version from 0.5 to trunk. I suspect that the best
> approach might be to use jessie with the default compiler for recent Qpids
> and only use wheezy for older ones to provide legacy support.
>
>
> I'm no bash expert so the build.sh used by the Dockerfiles is a little bit
> monolithic and could do with something of a refactor into functions, but
> hopefully what it's doing should be fairly clear.
>
> Hope this is useful.
>
> Best regards,
>
> Frase
>
>
>
> On 13/06/17 15:13, Irina Boverman wrote:
>
>> Hi everyone,
>>
>> I would like to propose creating Docker images for Qpid components hosted
>> in Docker Hub, updated upon component release and maintained by the
>> project, and I would like to contribute to doing this.
>>
>> Availability of Qpid images will make it easier to consume/deploy Qpid
>> components and promote Qpid visibility.
>>
>> We can maintain docker scripts creating these images from the base OS
>> images and using Qpid installation methods consistent with the OS
>> distribution. A possible naming convention might be qpid/<component>/<OS>.
>> I registered the 'qpid' user on DockerHub to use if this seems reasonable.
>> For example, we could create qpid/dispatch/<OS> image, qpid/<broker>/<OS>
>> image, qpid/<client(s)>/<OS> image, etc. Initially I would look to support
>> Fedora/CentOS latest images and Qpid components as RPMs for them, then aim
>> to expand OS coverage for debian/Ubuntu/etc in the future.
>>
>> The goal would be to update Qpid images within a few days upon component
>> release (either directly or indirectly using yum/dnf from public
>> repositories). We could ask the Docker team to grant Qpid "official"
>> status
>> when images have been stabilized.
>> --
>> Regards, Irina.
>>
>>
>
Re: Proposal to create Docker images for Qpid components.
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
Oops, I've just realised that I accidentally PM'd this reply to Jakub
rather than the group (sorry Jakub, finger trouble!)
Re: "I think the small images have their pros and cons. When I run into
problems I find it always great to have something like CentOS, Fedora or
Ubuntu as the base image. You can easily install many tools to debug the
problems etc."
Sure there are trades. With what I did the build system Dockerfile
actually creates an executable qpidd too so you *could* use that and do
the sort of things you allude to too, but I prefer to create minimal
images as I feel it's a cleaner approach and I'd much rather operational
images expose as minimal a surface as possible.
I'd generally go with Alpine as a base OS if I had to use a base OS, but
I've not got Qpid compiling on Alpine yet and as I managed to get a
minimal OS-less Image for qpidd working I didn't pursue it.
IMHO there are far too many fat, bloaty and "kind-of-random" images
floating around on Docker Hub and have a hunch all those miscellaneous,
and usually unmaintained and unpatched base images sitting atop
ostensibly official application images is an accident waiting to happen.
As I said in my reply to Irina I think that there's a lot to be said for
having well understood provenance and good versioning, which is
something I see precious little of on Docker Hub, call me old fashioned .
Frase
On 13/06/17 20:39, Jakub Scholz wrote:
> I think the small images have their pros and cons. When I run into problems
> I find it always great to have something like CentOS, Fedora or Ubuntu as
> the base image. You can easily install many tools to debug the problems
> etc. And often the size doesn't have to be that big. It depends how minimal
> the base image is.
>
> BTW: I have also created my own build system (
> https://github.com/scholzj/build-qpid-cpp and
> https://github.com/scholzj/build-qpid-dispatch). The great thing is that
> with Docker you can run the build anywhere including TravisCI or CircleCI.
> I think this is another area which we might improve on as community. To
> have also some nightly RPMs etc.
>
> Regards
> Jakub
>
> On Tue, Jun 13, 2017 at 5:36 PM, Fraser Adams <fraser.adams@blueyonder.co.uk
>> wrote:
>> Hi Irina,
>>
>> I think that this is an excellent idea and I totally agree with the
>> sentiment "Availability of Qpid images will make it easier to
>> consume/deploy Qpid components and promote Qpid visibility." I think that
>> in 2017 it's really important for the project to be providing official
>> maintained images as otherwise we'll continue to see a fragmented
>> miscellany of images of unknown provenance growing up out of necessity. To
>> be fair that seems to be a general trend on Docker Hub for many projects,
>> but I personally tend to find it a bit frustrating and would prefer a bit
>> more control over provenance and versioning for things used on production
>> systems.
>>
>> I'd make a few comments though.
>>
>> 1. It's probably better to propose this on the users list as that tends to
>> have more traffic than the dev list. If you choose to repost on the users
>> list feel free to include this response (if it's useful).
>>
>> 2. I do wonder about the comment "We can maintain docker scripts creating
>> these images from the base OS images and using Qpid installation methods
>> consistent with the OS distribution" and "I would look to support
>> Fedora/CentOS latest images" and "expand OS coverage for debian/Ubuntu/etc".
>>
>> I can see why this might be the /obvious/ approach, but IMHO I think that
>> it's a somewhat poor anti-pattern for containerisation that is propagated
>> way too often on Docker Hub and tends to result in large bloaty images with
>> larger potential attack surfaces than necessary and in addition reinforces
>> a model that containers are somehow "lightweight VMs" rather than
>> containerised appliances. I very much prefer a micro-containerised approach
>> using an approach similar to that descrived here:
>> http://blog.oddbit.com/2015/02/05/creating-minimal-docker-images/ though
>> acknowledging that this approach adds some complexity to the build process
>> and requires a little care.
>>
>> Ultimately reducing image size from the 400-500MB range to something like
>> 55MB for qpidd feels like a good thing for a number of reasons, the
>> following article contains some interesting musings on the subject
>> https://opensolitude.com/2015/05/13/docker-images-should-be-small.html.
>>
>> It's a bit more complex for Python and Java based components, but I think
>> that basing components on lighter weight Alpine Linux based Java or Python
>> runtimes is probably the best way to go, I note that Docker Hub is (slowly)
>> moving towards Alpine and http://www.iron.io/ have built some uber tiny
>> Docker images for a wide range of languages see
>> https://github.com/iron-io/dockers and https://hub.docker.com/u/iron/
>>
>> 3. One question to ask is which Qpid versions would be supported by the
>> build system. Obviously one advantage of containerisation is the ability to
>> easily use multiple versions of an application in order to support
>> regression testing and potentially legacy capabilities in an operational
>> environment, but of course there is a price to be paid as older versions
>> run into difficulties with newer compilers, boost versions etc.
>>
>>
>> I had a bit of a go at creating a build system for qpidd, my intention has
>> been to do the same for other components, but I've been stuck for time, see
>> https://github.com/fadams/docker-qpid though the qpid-cpp link is the
>> only one with any useful content.
>>
>> The approach that I took was to create a Dockerised build system based on
>> Debian so that the only dependency to actually build the component is a
>> working Docker install and an Internet connection, the build system
>> installs the necessary dev packages then installs the requested qpid-cpp
>> source, runs the Qpid build then it exports the relevant shared libraries
>> etc. and dynamic linker in order that a separate Dockerfile can use these
>> to create a small image containing only what is necessary for qpidd to
>> execute.
>>
>> I created two alternative build systems, one based on Debian wheezy the
>> other based on jessie, the former was somewhat simpler as wheezy has
>> somewhat older gcc and boost packages so less "wrangling" was necessary to
>> build really old Qpid versions, with jessie I had to build gcc and boost
>> from source. If you only intent to build recent Qpid versions much of that
>> complexity is unnecessary as recent Qpid versions compile with recent gcc
>> and boost versions, but I'm a little bit of a glutton for punishment and it
>> seemed a bit more of a challenge to try and create something that could
>> build every qpidd version from 0.5 to trunk. I suspect that the best
>> approach might be to use jessie with the default compiler for recent Qpids
>> and only use wheezy for older ones to provide legacy support.
>>
>>
>> I'm no bash expert so the build.sh used by the Dockerfiles is a little bit
>> monolithic and could do with something of a refactor into functions, but
>> hopefully what it's doing should be fairly clear.
>>
>> Hope this is useful.
>>
>> Best regards,
>>
>> Frase
>>
>>
>>
>> On 13/06/17 15:13, Irina Boverman wrote:
>>
>>> Hi everyone,
>>>
>>> I would like to propose creating Docker images for Qpid components hosted
>>> in Docker Hub, updated upon component release and maintained by the
>>> project, and I would like to contribute to doing this.
>>>
>>> Availability of Qpid images will make it easier to consume/deploy Qpid
>>> components and promote Qpid visibility.
>>>
>>> We can maintain docker scripts creating these images from the base OS
>>> images and using Qpid installation methods consistent with the OS
>>> distribution. A possible naming convention might be qpid/<component>/<OS>.
>>> I registered the 'qpid' user on DockerHub to use if this seems reasonable.
>>> For example, we could create qpid/dispatch/<OS> image, qpid/<broker>/<OS>
>>> image, qpid/<client(s)>/<OS> image, etc. Initially I would look to support
>>> Fedora/CentOS latest images and Qpid components as RPMs for them, then aim
>>> to expand OS coverage for debian/Ubuntu/etc in the future.
>>>
>>> The goal would be to update Qpid images within a few days upon component
>>> release (either directly or indirectly using yum/dnf from public
>>> repositories). We could ask the Docker team to grant Qpid "official"
>>> status
>>> when images have been stabilized.
>>> --
>>> Regards, Irina.
>>>
>>>
Re: Proposal to create Docker images for Qpid components.
Posted by Jakub Scholz <ja...@scholz.cz>.
I think the small images have their pros and cons. When I run into problems
I find it always great to have something like CentOS, Fedora or Ubuntu as
the base image. You can easily install many tools to debug the problems
etc. And often the size doesn't have to be that big. It depends how minimal
the base image is.
BTW: I have also created my own build system (
https://github.com/scholzj/build-qpid-cpp and
https://github.com/scholzj/build-qpid-dispatch). The great thing is that
with Docker you can run the build anywhere including TravisCI or CircleCI.
I think this is another area which we might improve on as community. To
have also some nightly RPMs etc.
Regards
Jakub
On Tue, Jun 13, 2017 at 5:36 PM, Fraser Adams <fraser.adams@blueyonder.co.uk
> wrote:
> Hi Irina,
>
> I think that this is an excellent idea and I totally agree with the
> sentiment "Availability of Qpid images will make it easier to
> consume/deploy Qpid components and promote Qpid visibility." I think that
> in 2017 it's really important for the project to be providing official
> maintained images as otherwise we'll continue to see a fragmented
> miscellany of images of unknown provenance growing up out of necessity. To
> be fair that seems to be a general trend on Docker Hub for many projects,
> but I personally tend to find it a bit frustrating and would prefer a bit
> more control over provenance and versioning for things used on production
> systems.
>
> I'd make a few comments though.
>
> 1. It's probably better to propose this on the users list as that tends to
> have more traffic than the dev list. If you choose to repost on the users
> list feel free to include this response (if it's useful).
>
> 2. I do wonder about the comment "We can maintain docker scripts creating
> these images from the base OS images and using Qpid installation methods
> consistent with the OS distribution" and "I would look to support
> Fedora/CentOS latest images" and "expand OS coverage for debian/Ubuntu/etc".
>
> I can see why this might be the /obvious/ approach, but IMHO I think that
> it's a somewhat poor anti-pattern for containerisation that is propagated
> way too often on Docker Hub and tends to result in large bloaty images with
> larger potential attack surfaces than necessary and in addition reinforces
> a model that containers are somehow "lightweight VMs" rather than
> containerised appliances. I very much prefer a micro-containerised approach
> using an approach similar to that descrived here:
> http://blog.oddbit.com/2015/02/05/creating-minimal-docker-images/ though
> acknowledging that this approach adds some complexity to the build process
> and requires a little care.
>
> Ultimately reducing image size from the 400-500MB range to something like
> 55MB for qpidd feels like a good thing for a number of reasons, the
> following article contains some interesting musings on the subject
> https://opensolitude.com/2015/05/13/docker-images-should-be-small.html.
>
> It's a bit more complex for Python and Java based components, but I think
> that basing components on lighter weight Alpine Linux based Java or Python
> runtimes is probably the best way to go, I note that Docker Hub is (slowly)
> moving towards Alpine and http://www.iron.io/ have built some uber tiny
> Docker images for a wide range of languages see
> https://github.com/iron-io/dockers and https://hub.docker.com/u/iron/
>
> 3. One question to ask is which Qpid versions would be supported by the
> build system. Obviously one advantage of containerisation is the ability to
> easily use multiple versions of an application in order to support
> regression testing and potentially legacy capabilities in an operational
> environment, but of course there is a price to be paid as older versions
> run into difficulties with newer compilers, boost versions etc.
>
>
> I had a bit of a go at creating a build system for qpidd, my intention has
> been to do the same for other components, but I've been stuck for time, see
> https://github.com/fadams/docker-qpid though the qpid-cpp link is the
> only one with any useful content.
>
> The approach that I took was to create a Dockerised build system based on
> Debian so that the only dependency to actually build the component is a
> working Docker install and an Internet connection, the build system
> installs the necessary dev packages then installs the requested qpid-cpp
> source, runs the Qpid build then it exports the relevant shared libraries
> etc. and dynamic linker in order that a separate Dockerfile can use these
> to create a small image containing only what is necessary for qpidd to
> execute.
>
> I created two alternative build systems, one based on Debian wheezy the
> other based on jessie, the former was somewhat simpler as wheezy has
> somewhat older gcc and boost packages so less "wrangling" was necessary to
> build really old Qpid versions, with jessie I had to build gcc and boost
> from source. If you only intent to build recent Qpid versions much of that
> complexity is unnecessary as recent Qpid versions compile with recent gcc
> and boost versions, but I'm a little bit of a glutton for punishment and it
> seemed a bit more of a challenge to try and create something that could
> build every qpidd version from 0.5 to trunk. I suspect that the best
> approach might be to use jessie with the default compiler for recent Qpids
> and only use wheezy for older ones to provide legacy support.
>
>
> I'm no bash expert so the build.sh used by the Dockerfiles is a little bit
> monolithic and could do with something of a refactor into functions, but
> hopefully what it's doing should be fairly clear.
>
> Hope this is useful.
>
> Best regards,
>
> Frase
>
>
>
> On 13/06/17 15:13, Irina Boverman wrote:
>
>> Hi everyone,
>>
>> I would like to propose creating Docker images for Qpid components hosted
>> in Docker Hub, updated upon component release and maintained by the
>> project, and I would like to contribute to doing this.
>>
>> Availability of Qpid images will make it easier to consume/deploy Qpid
>> components and promote Qpid visibility.
>>
>> We can maintain docker scripts creating these images from the base OS
>> images and using Qpid installation methods consistent with the OS
>> distribution. A possible naming convention might be qpid/<component>/<OS>.
>> I registered the 'qpid' user on DockerHub to use if this seems reasonable.
>> For example, we could create qpid/dispatch/<OS> image, qpid/<broker>/<OS>
>> image, qpid/<client(s)>/<OS> image, etc. Initially I would look to support
>> Fedora/CentOS latest images and Qpid components as RPMs for them, then aim
>> to expand OS coverage for debian/Ubuntu/etc in the future.
>>
>> The goal would be to update Qpid images within a few days upon component
>> release (either directly or indirectly using yum/dnf from public
>> repositories). We could ask the Docker team to grant Qpid "official"
>> status
>> when images have been stabilized.
>> --
>> Regards, Irina.
>>
>>
>
Re: Proposal to create Docker images for Qpid components.
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
Hi Irina,
I think that this is an excellent idea and I totally agree with the
sentiment "Availability of Qpid images will make it easier to
consume/deploy Qpid components and promote Qpid visibility." I think
that in 2017 it's really important for the project to be providing
official maintained images as otherwise we'll continue to see a
fragmented miscellany of images of unknown provenance growing up out of
necessity. To be fair that seems to be a general trend on Docker Hub for
many projects, but I personally tend to find it a bit frustrating and
would prefer a bit more control over provenance and versioning for
things used on production systems.
I'd make a few comments though.
1. It's probably better to propose this on the users list as that tends
to have more traffic than the dev list. If you choose to repost on the
users list feel free to include this response (if it's useful).
2. I do wonder about the comment "We can maintain docker scripts
creating these images from the base OS images and using Qpid
installation methods consistent with the OS distribution" and "I would
look to support Fedora/CentOS latest images" and "expand OS coverage for
debian/Ubuntu/etc".
I can see why this might be the /obvious/ approach, but IMHO I think
that it's a somewhat poor anti-pattern for containerisation that is
propagated way too often on Docker Hub and tends to result in large
bloaty images with larger potential attack surfaces than necessary and
in addition reinforces a model that containers are somehow "lightweight
VMs" rather than containerised appliances. I very much prefer a
micro-containerised approach using an approach similar to that descrived
here: http://blog.oddbit.com/2015/02/05/creating-minimal-docker-images/
though acknowledging that this approach adds some complexity to the
build process and requires a little care.
Ultimately reducing image size from the 400-500MB range to something
like 55MB for qpidd feels like a good thing for a number of reasons, the
following article contains some interesting musings on the subject
https://opensolitude.com/2015/05/13/docker-images-should-be-small.html.
It's a bit more complex for Python and Java based components, but I
think that basing components on lighter weight Alpine Linux based Java
or Python runtimes is probably the best way to go, I note that Docker
Hub is (slowly) moving towards Alpine and http://www.iron.io/ have built
some uber tiny Docker images for a wide range of languages see
https://github.com/iron-io/dockers and https://hub.docker.com/u/iron/
3. One question to ask is which Qpid versions would be supported by the
build system. Obviously one advantage of containerisation is the ability
to easily use multiple versions of an application in order to support
regression testing and potentially legacy capabilities in an operational
environment, but of course there is a price to be paid as older versions
run into difficulties with newer compilers, boost versions etc.
I had a bit of a go at creating a build system for qpidd, my intention
has been to do the same for other components, but I've been stuck for
time, see https://github.com/fadams/docker-qpid though the qpid-cpp link
is the only one with any useful content.
The approach that I took was to create a Dockerised build system based
on Debian so that the only dependency to actually build the component is
a working Docker install and an Internet connection, the build system
installs the necessary dev packages then installs the requested qpid-cpp
source, runs the Qpid build then it exports the relevant shared
libraries etc. and dynamic linker in order that a separate Dockerfile
can use these to create a small image containing only what is necessary
for qpidd to execute.
I created two alternative build systems, one based on Debian wheezy the
other based on jessie, the former was somewhat simpler as wheezy has
somewhat older gcc and boost packages so less "wrangling" was necessary
to build really old Qpid versions, with jessie I had to build gcc and
boost from source. If you only intent to build recent Qpid versions much
of that complexity is unnecessary as recent Qpid versions compile with
recent gcc and boost versions, but I'm a little bit of a glutton for
punishment and it seemed a bit more of a challenge to try and create
something that could build every qpidd version from 0.5 to trunk. I
suspect that the best approach might be to use jessie with the default
compiler for recent Qpids and only use wheezy for older ones to provide
legacy support.
I'm no bash expert so the build.sh used by the Dockerfiles is a little
bit monolithic and could do with something of a refactor into functions,
but hopefully what it's doing should be fairly clear.
Hope this is useful.
Best regards,
Frase
On 13/06/17 15:13, Irina Boverman wrote:
> Hi everyone,
>
> I would like to propose creating Docker images for Qpid components hosted
> in Docker Hub, updated upon component release and maintained by the
> project, and I would like to contribute to doing this.
>
> Availability of Qpid images will make it easier to consume/deploy Qpid
> components and promote Qpid visibility.
>
> We can maintain docker scripts creating these images from the base OS
> images and using Qpid installation methods consistent with the OS
> distribution. A possible naming convention might be qpid/<component>/<OS>.
> I registered the 'qpid' user on DockerHub to use if this seems reasonable.
> For example, we could create qpid/dispatch/<OS> image, qpid/<broker>/<OS>
> image, qpid/<client(s)>/<OS> image, etc. Initially I would look to support
> Fedora/CentOS latest images and Qpid components as RPMs for them, then aim
> to expand OS coverage for debian/Ubuntu/etc in the future.
>
> The goal would be to update Qpid images within a few days upon component
> release (either directly or indirectly using yum/dnf from public
> repositories). We could ask the Docker team to grant Qpid "official" status
> when images have been stabilized.
> --
> Regards, Irina.
>
Re: Proposal to create Docker images for Qpid components.
Posted by Robbie Gemmell <ro...@gmail.com>.
On 19 June 2017 at 13:57, Rob Godfrey <ro...@gmail.com> wrote:
> On 19 June 2017 at 14:16, Robbie Gemmell <ro...@gmail.com> wrote:
>
>> I think docker images would be good to have.
>>
>>
> +1
>
>
>> Rather than reply to lots of little bits from various different mails,
>> I'm just going to gather some general comments/questions here based on
>> the previous mails.
>>
>> - If I understand correctly Irina's proposal is essentially to create
>> create RPMs/DEBs (for certain components at least, might not be as
>> useful/necessary to others, see later) and then create docker images
>> using them (configure yum/dnf/apt repo, call install?).
>
>
>
> If this is the proposal then does that also mean that these rpms/debs will
> also have some sort of official status?
I would say so yes, one of my later items was relating to that aspect.
> If so would we want to
> document/vote that as well?
>
>
>> To Jakub's
>> point around doing things such as nightly RPMs, that could form part
>> of it however we still wouldn't especially promote them to end users,
>
> much like the maven repo snapshots arent, as such interim builds are
>> only meant to be known to folks tracking/partipating in development as
>> we are only allowed to direct regular everyday users at project
>> releases.
>>
>>
> Agreed - IIUC it would actually be against Apache policy to make available
> any "official" binary that was not just a convenience build of a voted
> release.
>
Yep
>
>> - One thing I think worth some thought/discussion is versioning, e.g .
>> what would the docker tags offered be. It looks like some projects do
>> precise full-version tags to a specific 'major.minor.patch' release,
>> as well as more general 'minor version' and 'major version' tags
>> netting the latest release in a given stream. There might be some
>> related behaviour there to decide on, e.g if using the OS package
>> systems, would running an update result in moving to a newer component
>> release in the stream. For some of those tag formats it could, while
>> for others it should not.
>>
>> - If we do start offering images based on packages, then those
>> packages and the related images essentially become 'convenience
>> binaries' for the release that get created+tested by the project as
>> part of the components release process. For bug fixes and
>> functionality updates to be usable in images, there will need to be
>> matching component releases to allow updating their packages and in
>> turn images.
>
>
> It seems to me that discussion on use of OS packaging systems and our
> versioning within them is worthy of a separate discussion / proposal (and
> may be a pre-requisite for building docker images for some components, if
> that is the way we wish to go).
>
Yes the specifics of any particular package should also be discussed
seperately. I'd say we are still at the point of this discusssion that
things can be discussed in tandem with regard to Docker images and
whether they go that way.
>
>> I think this points to the project doing more [patch]
>> releases for some of our components (i.e. ones we only do source
>> release archives for), which I think would also be a very good thing.
>>
>
> +1
>
>
>>
>> - Its true that having multiple base OS options would take
>> maintenance. Only having one option on the other hand probably
>> involves some decisions as to which (for a given component? Perhaps
>> they might not all be the same?). I think this also probably gets into
>> considering what people are looking for when they use a docker image.
>> In the case of a server such as a broker some people might just want
>> the port(s) listening and doing stuff to use/try-out a particular
>> server and dont care what OS etc is under it, while others might care
>> that that its on a particular OS variant (e.g one they use for all
>> their other bits), and others still might only care that its as small
>> an image as possible (somewhat combining the previous behaviours to an
>> extent).
>>
>
> Agreed - are there any precedents in other Apache projects that we think it
> would be useful to follow?
>
I'm not sure there is any consistent precedent, more a variety of
different appaches. Some dont mention OS at all, ones choose one,
others have a couple with perhaps Alpine being there for low size. For
some it also looks like images, though 'Docker official', aren't
necessarily maintained by the projects but rather the Docker community
instead. Some others it seems dont have a direct set of images at all
but perhaps do see popular images for related offerings built on them.
>
>>
>> - To the comments around the Java broker, I don't think creating
>> packages for it is really necessary? From a quick look at some others
>> images it doesnt seem unusual to have a Dockerfile set up to pull the
>> existing binary release archive, verify its sigs, and
>> extract+configure it in an appropriate location.
>>
>
> This also make sense to me. As a consumer of Java components, the use of
> OS package systems would seem unnecessary here.
>
> Do we have a list of the components/combinations which we think it would
> make sense to produce/support docker images for?
>
> -- Rob
>
>
>>
>> Robbie
>>
>> On 13 June 2017 at 15:13, Irina Boverman <ib...@redhat.com> wrote:
>> > Hi everyone,
>> >
>> > I would like to propose creating Docker images for Qpid components hosted
>> > in Docker Hub, updated upon component release and maintained by the
>> > project, and I would like to contribute to doing this.
>> >
>> > Availability of Qpid images will make it easier to consume/deploy Qpid
>> > components and promote Qpid visibility.
>> >
>> > We can maintain docker scripts creating these images from the base OS
>> > images and using Qpid installation methods consistent with the OS
>> > distribution. A possible naming convention might be
>> qpid/<component>/<OS>.
>> > I registered the 'qpid' user on DockerHub to use if this seems
>> reasonable.
>> > For example, we could create qpid/dispatch/<OS> image, qpid/<broker>/<OS>
>> > image, qpid/<client(s)>/<OS> image, etc. Initially I would look to
>> support
>> > Fedora/CentOS latest images and Qpid components as RPMs for them, then
>> aim
>> > to expand OS coverage for debian/Ubuntu/etc in the future.
>> >
>> > The goal would be to update Qpid images within a few days upon component
>> > release (either directly or indirectly using yum/dnf from public
>> > repositories). We could ask the Docker team to grant Qpid "official"
>> status
>> > when images have been stabilized.
>> > --
>> > Regards, Irina.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: dev-help@qpid.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Proposal to create Docker images for Qpid components.
Posted by Rob Godfrey <ro...@gmail.com>.
On 19 June 2017 at 14:16, Robbie Gemmell <ro...@gmail.com> wrote:
> I think docker images would be good to have.
>
>
+1
> Rather than reply to lots of little bits from various different mails,
> I'm just going to gather some general comments/questions here based on
> the previous mails.
>
> - If I understand correctly Irina's proposal is essentially to create
> create RPMs/DEBs (for certain components at least, might not be as
> useful/necessary to others, see later) and then create docker images
> using them (configure yum/dnf/apt repo, call install?).
If this is the proposal then does that also mean that these rpms/debs will
also have some sort of official status? If so would we want to
document/vote that as well?
> To Jakub's
> point around doing things such as nightly RPMs, that could form part
> of it however we still wouldn't especially promote them to end users,
much like the maven repo snapshots arent, as such interim builds are
> only meant to be known to folks tracking/partipating in development as
> we are only allowed to direct regular everyday users at project
> releases.
>
>
Agreed - IIUC it would actually be against Apache policy to make available
any "official" binary that was not just a convenience build of a voted
release.
> - One thing I think worth some thought/discussion is versioning, e.g .
> what would the docker tags offered be. It looks like some projects do
> precise full-version tags to a specific 'major.minor.patch' release,
> as well as more general 'minor version' and 'major version' tags
> netting the latest release in a given stream. There might be some
> related behaviour there to decide on, e.g if using the OS package
> systems, would running an update result in moving to a newer component
> release in the stream. For some of those tag formats it could, while
> for others it should not.
>
> - If we do start offering images based on packages, then those
> packages and the related images essentially become 'convenience
> binaries' for the release that get created+tested by the project as
> part of the components release process. For bug fixes and
> functionality updates to be usable in images, there will need to be
> matching component releases to allow updating their packages and in
> turn images.
It seems to me that discussion on use of OS packaging systems and our
versioning within them is worthy of a separate discussion / proposal (and
may be a pre-requisite for building docker images for some components, if
that is the way we wish to go).
> I think this points to the project doing more [patch]
> releases for some of our components (i.e. ones we only do source
> release archives for), which I think would also be a very good thing.
>
+1
>
> - Its true that having multiple base OS options would take
> maintenance. Only having one option on the other hand probably
> involves some decisions as to which (for a given component? Perhaps
> they might not all be the same?). I think this also probably gets into
> considering what people are looking for when they use a docker image.
> In the case of a server such as a broker some people might just want
> the port(s) listening and doing stuff to use/try-out a particular
> server and dont care what OS etc is under it, while others might care
> that that its on a particular OS variant (e.g one they use for all
> their other bits), and others still might only care that its as small
> an image as possible (somewhat combining the previous behaviours to an
> extent).
>
Agreed - are there any precedents in other Apache projects that we think it
would be useful to follow?
>
> - To the comments around the Java broker, I don't think creating
> packages for it is really necessary? From a quick look at some others
> images it doesnt seem unusual to have a Dockerfile set up to pull the
> existing binary release archive, verify its sigs, and
> extract+configure it in an appropriate location.
>
This also make sense to me. As a consumer of Java components, the use of
OS package systems would seem unnecessary here.
Do we have a list of the components/combinations which we think it would
make sense to produce/support docker images for?
-- Rob
>
> Robbie
>
> On 13 June 2017 at 15:13, Irina Boverman <ib...@redhat.com> wrote:
> > Hi everyone,
> >
> > I would like to propose creating Docker images for Qpid components hosted
> > in Docker Hub, updated upon component release and maintained by the
> > project, and I would like to contribute to doing this.
> >
> > Availability of Qpid images will make it easier to consume/deploy Qpid
> > components and promote Qpid visibility.
> >
> > We can maintain docker scripts creating these images from the base OS
> > images and using Qpid installation methods consistent with the OS
> > distribution. A possible naming convention might be
> qpid/<component>/<OS>.
> > I registered the 'qpid' user on DockerHub to use if this seems
> reasonable.
> > For example, we could create qpid/dispatch/<OS> image, qpid/<broker>/<OS>
> > image, qpid/<client(s)>/<OS> image, etc. Initially I would look to
> support
> > Fedora/CentOS latest images and Qpid components as RPMs for them, then
> aim
> > to expand OS coverage for debian/Ubuntu/etc in the future.
> >
> > The goal would be to update Qpid images within a few days upon component
> > release (either directly or indirectly using yum/dnf from public
> > repositories). We could ask the Docker team to grant Qpid "official"
> status
> > when images have been stabilized.
> > --
> > Regards, Irina.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> For additional commands, e-mail: dev-help@qpid.apache.org
>
>
Re: Proposal to create Docker images for Qpid components.
Posted by Justin Ross <ju...@gmail.com>.
On Thu, Jun 29, 2017 at 9:52 AM, Irina Boverman <ib...@redhat.com> wrote:
> Thank you all who thoughtfully responded to this proposal. This is a
> summary as I understood it and my comments regarding these points.
>
> (1) Qpid community is in favour of creating and maintaining docker images
> for cpp/java brokers
> and dispatch router. OS choice is less important, as long as the user can
> connect to the broker/router port. We only need to support one OS choice.
>
> I think this is a good start. It covers a use case when someone wants to
> test or deploy client apps against broker/router conveniently running as a
> docker service.
>
> I think there might be another use case for those users who want to develop
> client apps in a known environment with client libraries already installed
> for them to work with. I think the OS choice is important to these users
> because they may have a preference/better knowledge of this particular OS.
> Please consider this use case.
>
I'm personally interested in doing something for clients. It makes it very
easy for folks to evaluate them. This would definitely be a lower priority
than the servers, however.
> (2) Docker images should be of minimal size.
>
> As a concept, I fully agree with it. However, community support for
> official images also makes sense to me. These images go through review and
> testing, as well as, get patches for CVEs. There is a whole infrastructure
> in place in Fedora to build docker images, etc following best practices
> from OCI.
>
> Official images range in size between 120 to 200 mb, and have 10+ mil of
> downloads, so the size does not appear to be an obstacle to adopting them.
> It is my opinion that future official images will evolve to be better
> aligned with how they are used, for example "server" and "workstation",
> etc.
>
> REPOSITORY TAG IMAGE ID
> CREATED SIZE
> docker.io/ubuntu latest ebcd9d4fca80
> 6 weeks ago 117.9 MB
> docker.io/debian latest e5599115b6a6
> 5 months ago 123 MB
> docker.io/centos latest 67591570dd29
> 6 months ago 191.8 MB
> docker.io/fedora latest a1e614f0f30e
> 6 months ago 197.1 MB
>
> We can also modify the base image to remove some items if we think they are
> not needed.
>
> My personal preference is to start with fedora image because this
> distribution has latest upstream releases for qpid packages, and I am the
> maintainer for a number of these packages.
> Fedora community is also working on its own docker registry (it will have
> stable and test registries), has koji support for building images and
> guidelines for naming and versioning them.
>
Fedora and others have purpose-built minimal base images for this. I think
we should target those.
https://fedoraproject.org/wiki/Changes/ContainerMinimalImage
Re: Proposal to create Docker images for Qpid components.
Posted by Irina Boverman <ib...@redhat.com>.
For review and comments:
https://github.com/irinabov/docker-qpid-cpp-broker
https://github.com/irinabov/docker-qpid-dispatch-router
--
Regards, Irina.
On Thu, Dec 7, 2017 at 3:35 PM, Ken Giusti <kg...@redhat.com> wrote:
> Hey!
>
> Just following up on this thread. What's the status?
>
> thanks,
>
> -K
>
> On Tue, Jul 11, 2017 at 11:27 AM, Robbie Gemmell
> <ro...@gmail.com> wrote:
> > On 29 June 2017 at 17:52, Irina Boverman <ib...@redhat.com> wrote:
> >> Thank you all who thoughtfully responded to this proposal. This is a
> >> summary as I understood it and my comments regarding these points.
> >>
> >> (1) Qpid community is in favour of creating and maintaining docker
> images
> >> for cpp/java brokers
> >> and dispatch router. OS choice is less important, as long as the user
> can
> >> connect to the broker/router port. We only need to support one OS
> choice.
> >>
> >> I think this is a good start. It covers a use case when someone wants to
> >> test or deploy client apps against broker/router conveniently running
> as a
> >> docker service.
> >>
> >> I think there might be another use case for those users who want to
> develop
> >> client apps in a known environment with client libraries already
> installed
> >> for them to work with. I think the OS choice is important to these users
> >> because they may have a preference/better knowledge of this particular
> OS.
> >> Please consider this use case.
> >>
> >> (2) Docker images should be of minimal size.
> >>
> >> As a concept, I fully agree with it. However, community support for
> >> official images also makes sense to me. These images go through review
> and
> >> testing, as well as, get patches for CVEs. There is a whole
> infrastructure
> >> in place in Fedora to build docker images, etc following best practices
> >> from OCI.
> >>
> >> Official images range in size between 120 to 200 mb, and have 10+ mil of
> >> downloads, so the size does not appear to be an obstacle to adopting
> them.
> >> It is my opinion that future official images will evolve to be better
> >> aligned with how they are used, for example "server" and "workstation",
> etc.
> >>
> >> REPOSITORY TAG IMAGE ID
> >> CREATED SIZE
> >> docker.io/ubuntu latest ebcd9d4fca80
> >> 6 weeks ago 117.9 MB
> >> docker.io/debian latest e5599115b6a6
> >> 5 months ago 123 MB
> >> docker.io/centos latest 67591570dd29
> >> 6 months ago 191.8 MB
> >> docker.io/fedora latest a1e614f0f30e
> >> 6 months ago 197.1 MB
> >>
> >> We can also modify the base image to remove some items if we think they
> are
> >> not needed.
> >>
> >
> > I think using a base OS image is fine, and your text and Justin's link
> > suggests there are likely to be improvements in some regards in this
> > area too. There is also nothing stopping a further image as minimal
> > size option based on Alpine etc, but obviously components would need
> > to work and be supported there first and it wasn't clear that is
> > currently always true.
> >
> >> My personal preference is to start with fedora image because this
> >> distribution has latest upstream releases for qpid packages, and I am
> the
> >> maintainer for a number of these packages.
> >
> > I think we possibly need some clarity around the end result here, as
> > it could strongly influence this particular area.
> >
> > For images to be considered 'official' Apache Qpid artifacts
> > maintained directly by the project, they and their Qpid related
> > contents would really need to be fully managed in concert with the
> > projects release process. For example, if such an image were utilising
> > packages to incorporate their relevant Qpid bits then I think it
> > follows those packages would similarly need to be considered
> > 'official' and handled similarly. Utilising the Qpid packages from
> > Fedora distributions for example, wouldn't be appropriate in an
> > 'official Apache Qpid' image since they are not directly managed
> > artifacts of the Apache Qpid project.
> >
> > There wouldn't be an issue in simply considering such images to be a
> > community-created convenience binaries based on the projects source
> > release, as e.g. the existing Fedora packages themselves already are,
> > but they wouldn't be considered 'official' Apache Qpid artifacts. It
> > is worth saying this isn't particularly unusual, for example httpd
> > direct people to their source releases but do reference various third
> > party binaries for Windows users, and don't seem to be involved with
> > the docker images for httpd, with members of the docker community
> > instead handling that.
> >
> > I think either approach works here, having distribution based packages
> > obviously has certain advantages for users and the project, so its
> > probably more of a question around what people are actually expecting
> > and/or looking to contribute towards here.
> >
> >> Fedora community is also working on its own docker registry (it will
> have
> >> stable and test registries), has koji support for building images and
> >> guidelines for naming and versioning them.
> >>
> >> (3). What upstream releases will be dockerised?
> >>
> >> I would like to propose to start with latest released version. We will
> >> maintain at most 1 version at any time. All code necessary to create
> these
> >> images should be included in the upstream source repository starting
> with
> >> the next release. We will provide security patches for it as needed (and
> >> possibly will need patch releases). I am not sure if it will be
> possible to
> >> remove older releases from docker registries (TBD).
> >>
> >> (4) What will be used to configure docker images?
> >>
> >> We can use env to configure brokers/router, similar to what was done
> here:
> >> https://hub.docker.com/r/scholzj/qpid-cpp/ (courtesy of Jakub Scholz).
> >>
> >> References:
> >>
> >> #Changes/Layered Docker Image Build Service
> >> https://fedoraproject.org/wiki/Changes/Layered_Docker_
> Image_Build_Service#Policies_and_guidelines
> >>
> >> #Fedora Docker Layered Image Build Service by Adam Miller
> >> https://blog.openshift.com/wp-content/uploads/Introducing-
> Docker-Layered-Image-Build-Service.pdf
> >>
> >> #OpenShift Commons Briefing #57: Fedora Docker Layered Image Build
> Service
> >> by Adam Miller
> >> https://www.youtube.com/watch?v=HHm0L6Fw5nk
> >>
> >> #Layered Image Build System
> >> https://docs.pagure.org/infra-docs/sysadmin-guide/sops/
> layered-image-buildsys.html
> >>
> >> #Atomic Reactor: Python library with command line interface for building
> >> docker images.
> >> https://github.com/projectatomic/atomic-reactor
> >>
> >> #Fedora Docker Layered image build service now available blog
> >> https://communityblog.fedoraproject.org/fedora-
> docker-layered-image-build-service-now-available/
> >>
> >> #Example of container build: cockpit
> >> https://koji.fedoraproject.org/koji/packageinfo?packageID=17959
> >>
> >> #Container:Guidelines
> >> https://fedoraproject.org/wiki/Container:Guidelines
> >>
> >> #Fedora container repo
> >> https://src.fedoraproject.org/cgit/container/
> >>
> >> #Building a modular Linux OS
> >> https://docs.pagure.org/modularity/
> >>
> >> #projectatomic/container-best-practices
> >> https://github.com/projectatomic/container-best-practices
> >>
> >> #Container Best Practices
> >> http://docs.projectatomic.io/container-best-practices/
> >>
> >> #Red Hat Image Naming Policy
> >> https://github.com/projectatomic/ContainerApplicationGenericLab
> els/blob/master/vendor/redhat/names.md
> >>
> >> #Guidelines for Naming Fedora Packages (including docker images)
> >> https://fedoraproject.org/wiki/Packaging:Naming?rd=
> Packaging:NamingGuidelines
> >>
> >> #OCI: open container initiative
> >> https://www.opencontainers.org/
> >>
> >> #Changes/FedoraDockerRegistry
> >> https://fedoraproject.org/wiki/Changes/FedoraDockerRegistry
> >>
> >> #Creating minimal Docker images from dynamically linked ELF binaries
> >> http://blog.oddbit.com/2015/02/05/creating-minimal-docker-images/
> >>
> >> #Why docker images should be small
> >> https://opensolitude.com/2015/05/13/docker-images-should-be-small.html
> >>
> >> Regards, Irina.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> > For additional commands, e-mail: dev-help@qpid.apache.org
> >
>
>
>
> --
> -K
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> For additional commands, e-mail: dev-help@qpid.apache.org
>
>
Re: Proposal to create Docker images for Qpid components.
Posted by Ken Giusti <kg...@redhat.com>.
Hey!
Just following up on this thread. What's the status?
thanks,
-K
On Tue, Jul 11, 2017 at 11:27 AM, Robbie Gemmell
<ro...@gmail.com> wrote:
> On 29 June 2017 at 17:52, Irina Boverman <ib...@redhat.com> wrote:
>> Thank you all who thoughtfully responded to this proposal. This is a
>> summary as I understood it and my comments regarding these points.
>>
>> (1) Qpid community is in favour of creating and maintaining docker images
>> for cpp/java brokers
>> and dispatch router. OS choice is less important, as long as the user can
>> connect to the broker/router port. We only need to support one OS choice.
>>
>> I think this is a good start. It covers a use case when someone wants to
>> test or deploy client apps against broker/router conveniently running as a
>> docker service.
>>
>> I think there might be another use case for those users who want to develop
>> client apps in a known environment with client libraries already installed
>> for them to work with. I think the OS choice is important to these users
>> because they may have a preference/better knowledge of this particular OS.
>> Please consider this use case.
>>
>> (2) Docker images should be of minimal size.
>>
>> As a concept, I fully agree with it. However, community support for
>> official images also makes sense to me. These images go through review and
>> testing, as well as, get patches for CVEs. There is a whole infrastructure
>> in place in Fedora to build docker images, etc following best practices
>> from OCI.
>>
>> Official images range in size between 120 to 200 mb, and have 10+ mil of
>> downloads, so the size does not appear to be an obstacle to adopting them.
>> It is my opinion that future official images will evolve to be better
>> aligned with how they are used, for example "server" and "workstation", etc.
>>
>> REPOSITORY TAG IMAGE ID
>> CREATED SIZE
>> docker.io/ubuntu latest ebcd9d4fca80
>> 6 weeks ago 117.9 MB
>> docker.io/debian latest e5599115b6a6
>> 5 months ago 123 MB
>> docker.io/centos latest 67591570dd29
>> 6 months ago 191.8 MB
>> docker.io/fedora latest a1e614f0f30e
>> 6 months ago 197.1 MB
>>
>> We can also modify the base image to remove some items if we think they are
>> not needed.
>>
>
> I think using a base OS image is fine, and your text and Justin's link
> suggests there are likely to be improvements in some regards in this
> area too. There is also nothing stopping a further image as minimal
> size option based on Alpine etc, but obviously components would need
> to work and be supported there first and it wasn't clear that is
> currently always true.
>
>> My personal preference is to start with fedora image because this
>> distribution has latest upstream releases for qpid packages, and I am the
>> maintainer for a number of these packages.
>
> I think we possibly need some clarity around the end result here, as
> it could strongly influence this particular area.
>
> For images to be considered 'official' Apache Qpid artifacts
> maintained directly by the project, they and their Qpid related
> contents would really need to be fully managed in concert with the
> projects release process. For example, if such an image were utilising
> packages to incorporate their relevant Qpid bits then I think it
> follows those packages would similarly need to be considered
> 'official' and handled similarly. Utilising the Qpid packages from
> Fedora distributions for example, wouldn't be appropriate in an
> 'official Apache Qpid' image since they are not directly managed
> artifacts of the Apache Qpid project.
>
> There wouldn't be an issue in simply considering such images to be a
> community-created convenience binaries based on the projects source
> release, as e.g. the existing Fedora packages themselves already are,
> but they wouldn't be considered 'official' Apache Qpid artifacts. It
> is worth saying this isn't particularly unusual, for example httpd
> direct people to their source releases but do reference various third
> party binaries for Windows users, and don't seem to be involved with
> the docker images for httpd, with members of the docker community
> instead handling that.
>
> I think either approach works here, having distribution based packages
> obviously has certain advantages for users and the project, so its
> probably more of a question around what people are actually expecting
> and/or looking to contribute towards here.
>
>> Fedora community is also working on its own docker registry (it will have
>> stable and test registries), has koji support for building images and
>> guidelines for naming and versioning them.
>>
>> (3). What upstream releases will be dockerised?
>>
>> I would like to propose to start with latest released version. We will
>> maintain at most 1 version at any time. All code necessary to create these
>> images should be included in the upstream source repository starting with
>> the next release. We will provide security patches for it as needed (and
>> possibly will need patch releases). I am not sure if it will be possible to
>> remove older releases from docker registries (TBD).
>>
>> (4) What will be used to configure docker images?
>>
>> We can use env to configure brokers/router, similar to what was done here:
>> https://hub.docker.com/r/scholzj/qpid-cpp/ (courtesy of Jakub Scholz).
>>
>> References:
>>
>> #Changes/Layered Docker Image Build Service
>> https://fedoraproject.org/wiki/Changes/Layered_Docker_Image_Build_Service#Policies_and_guidelines
>>
>> #Fedora Docker Layered Image Build Service by Adam Miller
>> https://blog.openshift.com/wp-content/uploads/Introducing-Docker-Layered-Image-Build-Service.pdf
>>
>> #OpenShift Commons Briefing #57: Fedora Docker Layered Image Build Service
>> by Adam Miller
>> https://www.youtube.com/watch?v=HHm0L6Fw5nk
>>
>> #Layered Image Build System
>> https://docs.pagure.org/infra-docs/sysadmin-guide/sops/layered-image-buildsys.html
>>
>> #Atomic Reactor: Python library with command line interface for building
>> docker images.
>> https://github.com/projectatomic/atomic-reactor
>>
>> #Fedora Docker Layered image build service now available blog
>> https://communityblog.fedoraproject.org/fedora-docker-layered-image-build-service-now-available/
>>
>> #Example of container build: cockpit
>> https://koji.fedoraproject.org/koji/packageinfo?packageID=17959
>>
>> #Container:Guidelines
>> https://fedoraproject.org/wiki/Container:Guidelines
>>
>> #Fedora container repo
>> https://src.fedoraproject.org/cgit/container/
>>
>> #Building a modular Linux OS
>> https://docs.pagure.org/modularity/
>>
>> #projectatomic/container-best-practices
>> https://github.com/projectatomic/container-best-practices
>>
>> #Container Best Practices
>> http://docs.projectatomic.io/container-best-practices/
>>
>> #Red Hat Image Naming Policy
>> https://github.com/projectatomic/ContainerApplicationGenericLabels/blob/master/vendor/redhat/names.md
>>
>> #Guidelines for Naming Fedora Packages (including docker images)
>> https://fedoraproject.org/wiki/Packaging:Naming?rd=Packaging:NamingGuidelines
>>
>> #OCI: open container initiative
>> https://www.opencontainers.org/
>>
>> #Changes/FedoraDockerRegistry
>> https://fedoraproject.org/wiki/Changes/FedoraDockerRegistry
>>
>> #Creating minimal Docker images from dynamically linked ELF binaries
>> http://blog.oddbit.com/2015/02/05/creating-minimal-docker-images/
>>
>> #Why docker images should be small
>> https://opensolitude.com/2015/05/13/docker-images-should-be-small.html
>>
>> Regards, Irina.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> For additional commands, e-mail: dev-help@qpid.apache.org
>
--
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Proposal to create Docker images for Qpid components.
Posted by Robbie Gemmell <ro...@gmail.com>.
On 29 June 2017 at 17:52, Irina Boverman <ib...@redhat.com> wrote:
> Thank you all who thoughtfully responded to this proposal. This is a
> summary as I understood it and my comments regarding these points.
>
> (1) Qpid community is in favour of creating and maintaining docker images
> for cpp/java brokers
> and dispatch router. OS choice is less important, as long as the user can
> connect to the broker/router port. We only need to support one OS choice.
>
> I think this is a good start. It covers a use case when someone wants to
> test or deploy client apps against broker/router conveniently running as a
> docker service.
>
> I think there might be another use case for those users who want to develop
> client apps in a known environment with client libraries already installed
> for them to work with. I think the OS choice is important to these users
> because they may have a preference/better knowledge of this particular OS.
> Please consider this use case.
>
> (2) Docker images should be of minimal size.
>
> As a concept, I fully agree with it. However, community support for
> official images also makes sense to me. These images go through review and
> testing, as well as, get patches for CVEs. There is a whole infrastructure
> in place in Fedora to build docker images, etc following best practices
> from OCI.
>
> Official images range in size between 120 to 200 mb, and have 10+ mil of
> downloads, so the size does not appear to be an obstacle to adopting them.
> It is my opinion that future official images will evolve to be better
> aligned with how they are used, for example "server" and "workstation", etc.
>
> REPOSITORY TAG IMAGE ID
> CREATED SIZE
> docker.io/ubuntu latest ebcd9d4fca80
> 6 weeks ago 117.9 MB
> docker.io/debian latest e5599115b6a6
> 5 months ago 123 MB
> docker.io/centos latest 67591570dd29
> 6 months ago 191.8 MB
> docker.io/fedora latest a1e614f0f30e
> 6 months ago 197.1 MB
>
> We can also modify the base image to remove some items if we think they are
> not needed.
>
I think using a base OS image is fine, and your text and Justin's link
suggests there are likely to be improvements in some regards in this
area too. There is also nothing stopping a further image as minimal
size option based on Alpine etc, but obviously components would need
to work and be supported there first and it wasn't clear that is
currently always true.
> My personal preference is to start with fedora image because this
> distribution has latest upstream releases for qpid packages, and I am the
> maintainer for a number of these packages.
I think we possibly need some clarity around the end result here, as
it could strongly influence this particular area.
For images to be considered 'official' Apache Qpid artifacts
maintained directly by the project, they and their Qpid related
contents would really need to be fully managed in concert with the
projects release process. For example, if such an image were utilising
packages to incorporate their relevant Qpid bits then I think it
follows those packages would similarly need to be considered
'official' and handled similarly. Utilising the Qpid packages from
Fedora distributions for example, wouldn't be appropriate in an
'official Apache Qpid' image since they are not directly managed
artifacts of the Apache Qpid project.
There wouldn't be an issue in simply considering such images to be a
community-created convenience binaries based on the projects source
release, as e.g. the existing Fedora packages themselves already are,
but they wouldn't be considered 'official' Apache Qpid artifacts. It
is worth saying this isn't particularly unusual, for example httpd
direct people to their source releases but do reference various third
party binaries for Windows users, and don't seem to be involved with
the docker images for httpd, with members of the docker community
instead handling that.
I think either approach works here, having distribution based packages
obviously has certain advantages for users and the project, so its
probably more of a question around what people are actually expecting
and/or looking to contribute towards here.
> Fedora community is also working on its own docker registry (it will have
> stable and test registries), has koji support for building images and
> guidelines for naming and versioning them.
>
> (3). What upstream releases will be dockerised?
>
> I would like to propose to start with latest released version. We will
> maintain at most 1 version at any time. All code necessary to create these
> images should be included in the upstream source repository starting with
> the next release. We will provide security patches for it as needed (and
> possibly will need patch releases). I am not sure if it will be possible to
> remove older releases from docker registries (TBD).
>
> (4) What will be used to configure docker images?
>
> We can use env to configure brokers/router, similar to what was done here:
> https://hub.docker.com/r/scholzj/qpid-cpp/ (courtesy of Jakub Scholz).
>
> References:
>
> #Changes/Layered Docker Image Build Service
> https://fedoraproject.org/wiki/Changes/Layered_Docker_Image_Build_Service#Policies_and_guidelines
>
> #Fedora Docker Layered Image Build Service by Adam Miller
> https://blog.openshift.com/wp-content/uploads/Introducing-Docker-Layered-Image-Build-Service.pdf
>
> #OpenShift Commons Briefing #57: Fedora Docker Layered Image Build Service
> by Adam Miller
> https://www.youtube.com/watch?v=HHm0L6Fw5nk
>
> #Layered Image Build System
> https://docs.pagure.org/infra-docs/sysadmin-guide/sops/layered-image-buildsys.html
>
> #Atomic Reactor: Python library with command line interface for building
> docker images.
> https://github.com/projectatomic/atomic-reactor
>
> #Fedora Docker Layered image build service now available blog
> https://communityblog.fedoraproject.org/fedora-docker-layered-image-build-service-now-available/
>
> #Example of container build: cockpit
> https://koji.fedoraproject.org/koji/packageinfo?packageID=17959
>
> #Container:Guidelines
> https://fedoraproject.org/wiki/Container:Guidelines
>
> #Fedora container repo
> https://src.fedoraproject.org/cgit/container/
>
> #Building a modular Linux OS
> https://docs.pagure.org/modularity/
>
> #projectatomic/container-best-practices
> https://github.com/projectatomic/container-best-practices
>
> #Container Best Practices
> http://docs.projectatomic.io/container-best-practices/
>
> #Red Hat Image Naming Policy
> https://github.com/projectatomic/ContainerApplicationGenericLabels/blob/master/vendor/redhat/names.md
>
> #Guidelines for Naming Fedora Packages (including docker images)
> https://fedoraproject.org/wiki/Packaging:Naming?rd=Packaging:NamingGuidelines
>
> #OCI: open container initiative
> https://www.opencontainers.org/
>
> #Changes/FedoraDockerRegistry
> https://fedoraproject.org/wiki/Changes/FedoraDockerRegistry
>
> #Creating minimal Docker images from dynamically linked ELF binaries
> http://blog.oddbit.com/2015/02/05/creating-minimal-docker-images/
>
> #Why docker images should be small
> https://opensolitude.com/2015/05/13/docker-images-should-be-small.html
>
> Regards, Irina.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Proposal to create Docker images for Qpid components.
Posted by Irina Boverman <ib...@redhat.com>.
Thank you all who thoughtfully responded to this proposal. This is a
summary as I understood it and my comments regarding these points.
(1) Qpid community is in favour of creating and maintaining docker images
for cpp/java brokers
and dispatch router. OS choice is less important, as long as the user can
connect to the broker/router port. We only need to support one OS choice.
I think this is a good start. It covers a use case when someone wants to
test or deploy client apps against broker/router conveniently running as a
docker service.
I think there might be another use case for those users who want to develop
client apps in a known environment with client libraries already installed
for them to work with. I think the OS choice is important to these users
because they may have a preference/better knowledge of this particular OS.
Please consider this use case.
(2) Docker images should be of minimal size.
As a concept, I fully agree with it. However, community support for
official images also makes sense to me. These images go through review and
testing, as well as, get patches for CVEs. There is a whole infrastructure
in place in Fedora to build docker images, etc following best practices
from OCI.
Official images range in size between 120 to 200 mb, and have 10+ mil of
downloads, so the size does not appear to be an obstacle to adopting them.
It is my opinion that future official images will evolve to be better
aligned with how they are used, for example "server" and "workstation", etc.
REPOSITORY TAG IMAGE ID
CREATED SIZE
docker.io/ubuntu latest ebcd9d4fca80
6 weeks ago 117.9 MB
docker.io/debian latest e5599115b6a6
5 months ago 123 MB
docker.io/centos latest 67591570dd29
6 months ago 191.8 MB
docker.io/fedora latest a1e614f0f30e
6 months ago 197.1 MB
We can also modify the base image to remove some items if we think they are
not needed.
My personal preference is to start with fedora image because this
distribution has latest upstream releases for qpid packages, and I am the
maintainer for a number of these packages.
Fedora community is also working on its own docker registry (it will have
stable and test registries), has koji support for building images and
guidelines for naming and versioning them.
(3). What upstream releases will be dockerised?
I would like to propose to start with latest released version. We will
maintain at most 1 version at any time. All code necessary to create these
images should be included in the upstream source repository starting with
the next release. We will provide security patches for it as needed (and
possibly will need patch releases). I am not sure if it will be possible to
remove older releases from docker registries (TBD).
(4) What will be used to configure docker images?
We can use env to configure brokers/router, similar to what was done here:
https://hub.docker.com/r/scholzj/qpid-cpp/ (courtesy of Jakub Scholz).
References:
#Changes/Layered Docker Image Build Service
https://fedoraproject.org/wiki/Changes/Layered_Docker_Image_Build_Service#Policies_and_guidelines
#Fedora Docker Layered Image Build Service by Adam Miller
https://blog.openshift.com/wp-content/uploads/Introducing-Docker-Layered-Image-Build-Service.pdf
#OpenShift Commons Briefing #57: Fedora Docker Layered Image Build Service
by Adam Miller
https://www.youtube.com/watch?v=HHm0L6Fw5nk
#Layered Image Build System
https://docs.pagure.org/infra-docs/sysadmin-guide/sops/layered-image-buildsys.html
#Atomic Reactor: Python library with command line interface for building
docker images.
https://github.com/projectatomic/atomic-reactor
#Fedora Docker Layered image build service now available blog
https://communityblog.fedoraproject.org/fedora-docker-layered-image-build-service-now-available/
#Example of container build: cockpit
https://koji.fedoraproject.org/koji/packageinfo?packageID=17959
#Container:Guidelines
https://fedoraproject.org/wiki/Container:Guidelines
#Fedora container repo
https://src.fedoraproject.org/cgit/container/
#Building a modular Linux OS
https://docs.pagure.org/modularity/
#projectatomic/container-best-practices
https://github.com/projectatomic/container-best-practices
#Container Best Practices
http://docs.projectatomic.io/container-best-practices/
#Red Hat Image Naming Policy
https://github.com/projectatomic/ContainerApplicationGenericLabels/blob/master/vendor/redhat/names.md
#Guidelines for Naming Fedora Packages (including docker images)
https://fedoraproject.org/wiki/Packaging:Naming?rd=Packaging:NamingGuidelines
#OCI: open container initiative
https://www.opencontainers.org/
#Changes/FedoraDockerRegistry
https://fedoraproject.org/wiki/Changes/FedoraDockerRegistry
#Creating minimal Docker images from dynamically linked ELF binaries
http://blog.oddbit.com/2015/02/05/creating-minimal-docker-images/
#Why docker images should be small
https://opensolitude.com/2015/05/13/docker-images-should-be-small.html
Regards, Irina.
RE: Proposal to create Docker images for Qpid components.
Posted by Steve Huston <sh...@riverace.com>.
I agree w/ Rob, Robbie... 2 brokers and dispatch. Clients don't make sense for docker images, IMO.
-Steve
> -----Original Message-----
> From: Rob Godfrey [mailto:rob.j.godfrey@gmail.com]
> Sent: Tuesday, June 20, 2017 1:19 PM
> To: qpid <de...@qpid.apache.org>
> Subject: Re: Proposal to create Docker images for Qpid components.
>
> On 20 June 2017 at 18:20, Robbie Gemmell <ro...@gmail.com>
> wrote:
>
> > I think it makes sense for qpid-broker-j, qpid-cpp broker, and
> > qpid-dispatch.
> >
>
> +1 These are what I imagined would make sense
>
>
> >
> > I believe some folks might also like 'client' images, which is much
> > less obvious to me..though I can see that for those needing
> > compilation or with interdependencies on bits that do, perhaps it
> > could make them slightly easier to get started with. Packages would
> > also in many cases though.
> >
> >
> Yeah - I'm not totally sold on the need for Docker images here (though I'm
> not necessarily against)... packages make a lot of sense to me however.
>
> -- Rob
>
>
> > On 20 June 2017 at 14:17, Rob Godfrey <ro...@gmail.com> wrote:
> > > So, stepping back for a second, which components do we think we
> > > should be releasing docker images for (and once we've agreed on this
> > > we can agree
> > on
> > > the number/form of images for each component perhaps :-) )?
> > >
> > > -- Rob
> > >
> > > On 20 June 2017 at 14:06, Robbie Gemmell <ro...@gmail.com>
> > wrote:
> > >
> > >> It was talking about downloading the built Java broker binary
> > >> release tar.gz, verifying it, and doing something with it. It
> > >> wasn't saying anything in particular about the OS, except there is
> > >> one and Java is available somehow.
> > >>
> > >> For example, some randomly selected 'docker official' images I
> > >> looked at for Apache projects with Java components which all
> > >> happened to do this (I'm sure there are others that are different, of
> course):
> > >>
> > >> https://hub.docker.com/r/_/tomcat/
> > >> not-alpine: https://github.com/docker-library/tomcat/blob/
> > >> 5ac222d258dc70c77bb3a9a4fab81ea286c9abd1/8.5/jre8/Dockerfile
> > >> alpine: https://github.com/docker-library/tomcat/blob/
> > >> 5ac222d258dc70c77bb3a9a4fab81ea286c9abd1/8.5/jre8-
> alpine/Dockerfile
> > >>
> > >> https://hub.docker.com/_/maven/
> > >> not-alpine: https://github.com/carlossg/docker-maven/blob/
> > >> 0490eff01e529b2d94789511b008d01a7b314953/jdk-8/Dockerfile
> > >> alpine: https://github.com/carlossg/docker-maven/blob/
> > >> 2357d3394f19730172ac9c7f4afe7cf052f36b4d/jdk-8/Dockerfile
> > >>
> > >> https://hub.docker.com/_/zookeeper/
> > >> alpine: https://github.com/31z4/zookeeper-docker/blob/
> > >> f12428ab7c6ea263ef037cf258129b83276c009c/3.4.10/Dockerfile
> > >>
> > >> At another try I got one thats doing something different:
> > >>
> > >> https://hub.docker.com/_/cassandra/
> > >> https://github.com/docker-library/cassandra/blob/
> > >> d83b850cd17bc9198876f8686197c730e29c7448/3.10/Dockerfile
> > >>
> > >> Here they seem to be using their own .deb files via
> > >> http://www.apache.org/dist/cassandra/debian which actually
> > >> redirects to http://dl.bintray.com/apache/cassandra/, a debian repo
> > >> (https://bintray.com/apache/cassandra/debian) set up within the ASF
> > >> org on bintray (https://bintray.com/apache)
> > >>
> > >> Robbie
> > >>
> > >> On 20 June 2017 at 11:05, Fraser Adams
> > >> <fr...@blueyonder.co.uk>
> > >> wrote:
> > >> > Re: "it doesnt seem unusual to have a Dockerfile set up to pull
> > >> > the
> > >> existing
> > >> > binary release archive, verify its sigs, and extract+configure it
> > >> > in
> > an
> > >> > appropriate location."
> > >> >
> > >> > Yes, it's certainly not unusual, but my personal view is that it
> > >> > isn't
> > >> great
> > >> > practice.
> > >> >
> > >> > As I said in my earlier reply to Irina, IMHO there are far too
> > >> > many instances of really bloaty Docker images containing far more
> > >> > than they
> > >> need,
> > >> > as well as unnecessarily making images larger than they need to
> > >> > be
> > (which
> > >> > isn't great if you are doing Continuous Deployment on a large
> > >> > system)
> > it
> > >> > also unnecessarily increases the attack surface. Now OK Qpid
> > >> > brokers
> > are
> > >> > probably long-lived services, so the first point might about
> > minimising
> > >> size
> > >> > may apply less to them than say 12 Factor App business function
> > services,
> > >> > but as a general principle I tend to think that not enough
> > >> > thought is
> > >> given
> > >> > to the footprint of Docker images.
> > >> >
> > >> > I may have misunderstood, If the sentence I've quoted is
> > >> > referring to
> > a
> > >> > Dockerfile for a *build system*, which subsequently exports a zip
> > >> containing
> > >> > only that necessary to build (using a separate Dockerfile) a
> > >> > small, versioned microcontainer based on a minimal distro like
> > >> > Alpine (or
> > from
> > >> > scratch) then that's fine, but having an image intended for use
> > >> > on a production system doing that sort of thing doesn't seem
> > >> > appropriate to
> > >> me.
> > >> >
> > >> > F.
> > >> >
> > >> >
> > >> >
> > >> > On 20/06/17 08:54, Lorenz Quack wrote:
> > >> >>
> > >> >> On Mon, 2017-06-19 at 13:16 +0100, Robbie Gemmell wrote:
> > >> >>>
> > >> >>> - To the comments around the Java broker, I don't think
> > >> >>> creating packages for it is really necessary? From a quick look
> > >> >>> at some
> > others
> > >> >>> images it doesnt seem unusual to have a Dockerfile set up to
> > >> >>> pull
> > the
> > >> >>> existing binary release archive, verify its sigs, and
> > >> >>> extract+configure it in an appropriate location.
> > >> >>>
> > >> >> Great. That would work for me.
> > >> >> I just thought it would be good to have the entire Qpid project
> > >> >> represented and to provide some choice at the same time.
> > >> >>
> > >> >> Kind regards,
> > >> >> Lorenz
> > >> >>
> > >> >>> Robbie
> > >> >>>
> > >> >>> On 13 June 2017 at 15:13, Irina Boverman <ib...@redhat.com>
> > wrote:
> > >> >>>>
> > >> >>>> Hi everyone,
> > >> >>>>
> > >> >>>> I would like to propose creating Docker images for Qpid
> > >> >>>> components hosted in Docker Hub, updated upon component
> > >> >>>> release and maintained by the project, and I would like to
> > >> >>>> contribute to doing this.
> > >> >>>>
> > >> >>>> Availability of Qpid images will make it easier to
> > >> >>>> consume/deploy
> > Qpid
> > >> >>>> components and promote Qpid visibility.
> > >> >>>>
> > >> >>>> We can maintain docker scripts creating these images from the
> > >> >>>> base
> > OS
> > >> >>>> images and using Qpid installation methods consistent with the
> > >> >>>> OS distribution. A possible naming convention might be
> > >> >>>> qpid/<component>/<OS>.
> > >> >>>> I registered the 'qpid' user on DockerHub to use if this seems
> > >> >>>> reasonable.
> > >> >>>> For example, we could create qpid/dispatch/<OS> image,
> > >> >>>> qpid/<broker>/<OS> image, qpid/<client(s)>/<OS> image, etc.
> > >> >>>> Initially I would look to support Fedora/CentOS latest images
> > >> >>>> and Qpid components as RPMs for them,
> > then
> > >> >>>> aim
> > >> >>>> to expand OS coverage for debian/Ubuntu/etc in the future.
> > >> >>>>
> > >> >>>> The goal would be to update Qpid images within a few days upon
> > >> component
> > >> >>>> release (either directly or indirectly using yum/dnf from
> > >> >>>> public repositories). We could ask the Docker team to grant
> > >> >>>> Qpid
> > "official"
> > >> >>>> status
> > >> >>>> when images have been stabilized.
> > >> >>>> --
> > >> >>>> Regards, Irina.
> > >> >>>
> > >> >>> ------------------------------------------------------------
> > ---------
> > >> >>> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org For
> > >> >>> additional commands, e-mail: dev-help@qpid.apache.org
> > >> >>>
> > >> >> ------------------------------------------------------------
> > ---------
> > >> >> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org For
> > >> >> additional commands, e-mail: dev-help@qpid.apache.org
> > >> >>
> > >> >
> > >> >
> > >> > -----------------------------------------------------------------
> > >> > ---- To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org For
> > >> > additional commands, e-mail: dev-help@qpid.apache.org
> > >> >
> > >>
> > >> -------------------------------------------------------------------
> > >> -- To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org For
> > >> additional commands, e-mail: dev-help@qpid.apache.org
> > >>
> > >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org For additional
> > commands, e-mail: dev-help@qpid.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Proposal to create Docker images for Qpid components.
Posted by Rob Godfrey <ro...@gmail.com>.
On 20 June 2017 at 18:20, Robbie Gemmell <ro...@gmail.com> wrote:
> I think it makes sense for qpid-broker-j, qpid-cpp broker, and
> qpid-dispatch.
>
+1 These are what I imagined would make sense
>
> I believe some folks might also like 'client' images, which is much
> less obvious to me..though I can see that for those needing
> compilation or with interdependencies on bits that do, perhaps it
> could make them slightly easier to get started with. Packages would
> also in many cases though.
>
>
Yeah - I'm not totally sold on the need for Docker images here (though I'm
not necessarily against)... packages make a lot of sense to me however.
-- Rob
> On 20 June 2017 at 14:17, Rob Godfrey <ro...@gmail.com> wrote:
> > So, stepping back for a second, which components do we think we should be
> > releasing docker images for (and once we've agreed on this we can agree
> on
> > the number/form of images for each component perhaps :-) )?
> >
> > -- Rob
> >
> > On 20 June 2017 at 14:06, Robbie Gemmell <ro...@gmail.com>
> wrote:
> >
> >> It was talking about downloading the built Java broker binary release
> >> tar.gz, verifying it, and doing something with it. It wasn't saying
> >> anything in particular about the OS, except there is one and Java is
> >> available somehow.
> >>
> >> For example, some randomly selected 'docker official' images I looked
> >> at for Apache projects with Java components which all happened to do
> >> this (I'm sure there are others that are different, of course):
> >>
> >> https://hub.docker.com/r/_/tomcat/
> >> not-alpine: https://github.com/docker-library/tomcat/blob/
> >> 5ac222d258dc70c77bb3a9a4fab81ea286c9abd1/8.5/jre8/Dockerfile
> >> alpine: https://github.com/docker-library/tomcat/blob/
> >> 5ac222d258dc70c77bb3a9a4fab81ea286c9abd1/8.5/jre8-alpine/Dockerfile
> >>
> >> https://hub.docker.com/_/maven/
> >> not-alpine: https://github.com/carlossg/docker-maven/blob/
> >> 0490eff01e529b2d94789511b008d01a7b314953/jdk-8/Dockerfile
> >> alpine: https://github.com/carlossg/docker-maven/blob/
> >> 2357d3394f19730172ac9c7f4afe7cf052f36b4d/jdk-8/Dockerfile
> >>
> >> https://hub.docker.com/_/zookeeper/
> >> alpine: https://github.com/31z4/zookeeper-docker/blob/
> >> f12428ab7c6ea263ef037cf258129b83276c009c/3.4.10/Dockerfile
> >>
> >> At another try I got one thats doing something different:
> >>
> >> https://hub.docker.com/_/cassandra/
> >> https://github.com/docker-library/cassandra/blob/
> >> d83b850cd17bc9198876f8686197c730e29c7448/3.10/Dockerfile
> >>
> >> Here they seem to be using their own .deb files via
> >> http://www.apache.org/dist/cassandra/debian which actually redirects
> >> to http://dl.bintray.com/apache/cassandra/, a debian repo
> >> (https://bintray.com/apache/cassandra/debian) set up within the ASF
> >> org on bintray (https://bintray.com/apache)
> >>
> >> Robbie
> >>
> >> On 20 June 2017 at 11:05, Fraser Adams <fr...@blueyonder.co.uk>
> >> wrote:
> >> > Re: "it doesnt seem unusual to have a Dockerfile set up to pull the
> >> existing
> >> > binary release archive, verify its sigs, and extract+configure it in
> an
> >> > appropriate location."
> >> >
> >> > Yes, it's certainly not unusual, but my personal view is that it isn't
> >> great
> >> > practice.
> >> >
> >> > As I said in my earlier reply to Irina, IMHO there are far too many
> >> > instances of really bloaty Docker images containing far more than they
> >> need,
> >> > as well as unnecessarily making images larger than they need to be
> (which
> >> > isn't great if you are doing Continuous Deployment on a large system)
> it
> >> > also unnecessarily increases the attack surface. Now OK Qpid brokers
> are
> >> > probably long-lived services, so the first point might about
> minimising
> >> size
> >> > may apply less to them than say 12 Factor App business function
> services,
> >> > but as a general principle I tend to think that not enough thought is
> >> given
> >> > to the footprint of Docker images.
> >> >
> >> > I may have misunderstood, If the sentence I've quoted is referring to
> a
> >> > Dockerfile for a *build system*, which subsequently exports a zip
> >> containing
> >> > only that necessary to build (using a separate Dockerfile) a small,
> >> > versioned microcontainer based on a minimal distro like Alpine (or
> from
> >> > scratch) then that's fine, but having an image intended for use on a
> >> > production system doing that sort of thing doesn't seem appropriate to
> >> me.
> >> >
> >> > F.
> >> >
> >> >
> >> >
> >> > On 20/06/17 08:54, Lorenz Quack wrote:
> >> >>
> >> >> On Mon, 2017-06-19 at 13:16 +0100, Robbie Gemmell wrote:
> >> >>>
> >> >>> - To the comments around the Java broker, I don't think creating
> >> >>> packages for it is really necessary? From a quick look at some
> others
> >> >>> images it doesnt seem unusual to have a Dockerfile set up to pull
> the
> >> >>> existing binary release archive, verify its sigs, and
> >> >>> extract+configure it in an appropriate location.
> >> >>>
> >> >> Great. That would work for me.
> >> >> I just thought it would be good to have the entire Qpid project
> >> >> represented
> >> >> and to provide some choice at the same time.
> >> >>
> >> >> Kind regards,
> >> >> Lorenz
> >> >>
> >> >>> Robbie
> >> >>>
> >> >>> On 13 June 2017 at 15:13, Irina Boverman <ib...@redhat.com>
> wrote:
> >> >>>>
> >> >>>> Hi everyone,
> >> >>>>
> >> >>>> I would like to propose creating Docker images for Qpid components
> >> >>>> hosted
> >> >>>> in Docker Hub, updated upon component release and maintained by the
> >> >>>> project, and I would like to contribute to doing this.
> >> >>>>
> >> >>>> Availability of Qpid images will make it easier to consume/deploy
> Qpid
> >> >>>> components and promote Qpid visibility.
> >> >>>>
> >> >>>> We can maintain docker scripts creating these images from the base
> OS
> >> >>>> images and using Qpid installation methods consistent with the OS
> >> >>>> distribution. A possible naming convention might be
> >> >>>> qpid/<component>/<OS>.
> >> >>>> I registered the 'qpid' user on DockerHub to use if this seems
> >> >>>> reasonable.
> >> >>>> For example, we could create qpid/dispatch/<OS> image,
> >> >>>> qpid/<broker>/<OS>
> >> >>>> image, qpid/<client(s)>/<OS> image, etc. Initially I would look to
> >> >>>> support
> >> >>>> Fedora/CentOS latest images and Qpid components as RPMs for them,
> then
> >> >>>> aim
> >> >>>> to expand OS coverage for debian/Ubuntu/etc in the future.
> >> >>>>
> >> >>>> The goal would be to update Qpid images within a few days upon
> >> component
> >> >>>> release (either directly or indirectly using yum/dnf from public
> >> >>>> repositories). We could ask the Docker team to grant Qpid
> "official"
> >> >>>> status
> >> >>>> when images have been stabilized.
> >> >>>> --
> >> >>>> Regards, Irina.
> >> >>>
> >> >>> ------------------------------------------------------------
> ---------
> >> >>> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> >> >>> For additional commands, e-mail: dev-help@qpid.apache.org
> >> >>>
> >> >> ------------------------------------------------------------
> ---------
> >> >> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> >> >> For additional commands, e-mail: dev-help@qpid.apache.org
> >> >>
> >> >
> >> >
> >> > ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> >> > For additional commands, e-mail: dev-help@qpid.apache.org
> >> >
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> >> For additional commands, e-mail: dev-help@qpid.apache.org
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> For additional commands, e-mail: dev-help@qpid.apache.org
>
>
Re: Proposal to create Docker images for Qpid components.
Posted by Robbie Gemmell <ro...@gmail.com>.
I think it makes sense for qpid-broker-j, qpid-cpp broker, and qpid-dispatch.
I believe some folks might also like 'client' images, which is much
less obvious to me..though I can see that for those needing
compilation or with interdependencies on bits that do, perhaps it
could make them slightly easier to get started with. Packages would
also in many cases though.
On 20 June 2017 at 14:17, Rob Godfrey <ro...@gmail.com> wrote:
> So, stepping back for a second, which components do we think we should be
> releasing docker images for (and once we've agreed on this we can agree on
> the number/form of images for each component perhaps :-) )?
>
> -- Rob
>
> On 20 June 2017 at 14:06, Robbie Gemmell <ro...@gmail.com> wrote:
>
>> It was talking about downloading the built Java broker binary release
>> tar.gz, verifying it, and doing something with it. It wasn't saying
>> anything in particular about the OS, except there is one and Java is
>> available somehow.
>>
>> For example, some randomly selected 'docker official' images I looked
>> at for Apache projects with Java components which all happened to do
>> this (I'm sure there are others that are different, of course):
>>
>> https://hub.docker.com/r/_/tomcat/
>> not-alpine: https://github.com/docker-library/tomcat/blob/
>> 5ac222d258dc70c77bb3a9a4fab81ea286c9abd1/8.5/jre8/Dockerfile
>> alpine: https://github.com/docker-library/tomcat/blob/
>> 5ac222d258dc70c77bb3a9a4fab81ea286c9abd1/8.5/jre8-alpine/Dockerfile
>>
>> https://hub.docker.com/_/maven/
>> not-alpine: https://github.com/carlossg/docker-maven/blob/
>> 0490eff01e529b2d94789511b008d01a7b314953/jdk-8/Dockerfile
>> alpine: https://github.com/carlossg/docker-maven/blob/
>> 2357d3394f19730172ac9c7f4afe7cf052f36b4d/jdk-8/Dockerfile
>>
>> https://hub.docker.com/_/zookeeper/
>> alpine: https://github.com/31z4/zookeeper-docker/blob/
>> f12428ab7c6ea263ef037cf258129b83276c009c/3.4.10/Dockerfile
>>
>> At another try I got one thats doing something different:
>>
>> https://hub.docker.com/_/cassandra/
>> https://github.com/docker-library/cassandra/blob/
>> d83b850cd17bc9198876f8686197c730e29c7448/3.10/Dockerfile
>>
>> Here they seem to be using their own .deb files via
>> http://www.apache.org/dist/cassandra/debian which actually redirects
>> to http://dl.bintray.com/apache/cassandra/, a debian repo
>> (https://bintray.com/apache/cassandra/debian) set up within the ASF
>> org on bintray (https://bintray.com/apache)
>>
>> Robbie
>>
>> On 20 June 2017 at 11:05, Fraser Adams <fr...@blueyonder.co.uk>
>> wrote:
>> > Re: "it doesnt seem unusual to have a Dockerfile set up to pull the
>> existing
>> > binary release archive, verify its sigs, and extract+configure it in an
>> > appropriate location."
>> >
>> > Yes, it's certainly not unusual, but my personal view is that it isn't
>> great
>> > practice.
>> >
>> > As I said in my earlier reply to Irina, IMHO there are far too many
>> > instances of really bloaty Docker images containing far more than they
>> need,
>> > as well as unnecessarily making images larger than they need to be (which
>> > isn't great if you are doing Continuous Deployment on a large system) it
>> > also unnecessarily increases the attack surface. Now OK Qpid brokers are
>> > probably long-lived services, so the first point might about minimising
>> size
>> > may apply less to them than say 12 Factor App business function services,
>> > but as a general principle I tend to think that not enough thought is
>> given
>> > to the footprint of Docker images.
>> >
>> > I may have misunderstood, If the sentence I've quoted is referring to a
>> > Dockerfile for a *build system*, which subsequently exports a zip
>> containing
>> > only that necessary to build (using a separate Dockerfile) a small,
>> > versioned microcontainer based on a minimal distro like Alpine (or from
>> > scratch) then that's fine, but having an image intended for use on a
>> > production system doing that sort of thing doesn't seem appropriate to
>> me.
>> >
>> > F.
>> >
>> >
>> >
>> > On 20/06/17 08:54, Lorenz Quack wrote:
>> >>
>> >> On Mon, 2017-06-19 at 13:16 +0100, Robbie Gemmell wrote:
>> >>>
>> >>> - To the comments around the Java broker, I don't think creating
>> >>> packages for it is really necessary? From a quick look at some others
>> >>> images it doesnt seem unusual to have a Dockerfile set up to pull the
>> >>> existing binary release archive, verify its sigs, and
>> >>> extract+configure it in an appropriate location.
>> >>>
>> >> Great. That would work for me.
>> >> I just thought it would be good to have the entire Qpid project
>> >> represented
>> >> and to provide some choice at the same time.
>> >>
>> >> Kind regards,
>> >> Lorenz
>> >>
>> >>> Robbie
>> >>>
>> >>> On 13 June 2017 at 15:13, Irina Boverman <ib...@redhat.com> wrote:
>> >>>>
>> >>>> Hi everyone,
>> >>>>
>> >>>> I would like to propose creating Docker images for Qpid components
>> >>>> hosted
>> >>>> in Docker Hub, updated upon component release and maintained by the
>> >>>> project, and I would like to contribute to doing this.
>> >>>>
>> >>>> Availability of Qpid images will make it easier to consume/deploy Qpid
>> >>>> components and promote Qpid visibility.
>> >>>>
>> >>>> We can maintain docker scripts creating these images from the base OS
>> >>>> images and using Qpid installation methods consistent with the OS
>> >>>> distribution. A possible naming convention might be
>> >>>> qpid/<component>/<OS>.
>> >>>> I registered the 'qpid' user on DockerHub to use if this seems
>> >>>> reasonable.
>> >>>> For example, we could create qpid/dispatch/<OS> image,
>> >>>> qpid/<broker>/<OS>
>> >>>> image, qpid/<client(s)>/<OS> image, etc. Initially I would look to
>> >>>> support
>> >>>> Fedora/CentOS latest images and Qpid components as RPMs for them, then
>> >>>> aim
>> >>>> to expand OS coverage for debian/Ubuntu/etc in the future.
>> >>>>
>> >>>> The goal would be to update Qpid images within a few days upon
>> component
>> >>>> release (either directly or indirectly using yum/dnf from public
>> >>>> repositories). We could ask the Docker team to grant Qpid "official"
>> >>>> status
>> >>>> when images have been stabilized.
>> >>>> --
>> >>>> Regards, Irina.
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
>> >>> For additional commands, e-mail: dev-help@qpid.apache.org
>> >>>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
>> >> For additional commands, e-mail: dev-help@qpid.apache.org
>> >>
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
>> > For additional commands, e-mail: dev-help@qpid.apache.org
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: dev-help@qpid.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Proposal to create Docker images for Qpid components.
Posted by Rob Godfrey <ro...@gmail.com>.
So, stepping back for a second, which components do we think we should be
releasing docker images for (and once we've agreed on this we can agree on
the number/form of images for each component perhaps :-) )?
-- Rob
On 20 June 2017 at 14:06, Robbie Gemmell <ro...@gmail.com> wrote:
> It was talking about downloading the built Java broker binary release
> tar.gz, verifying it, and doing something with it. It wasn't saying
> anything in particular about the OS, except there is one and Java is
> available somehow.
>
> For example, some randomly selected 'docker official' images I looked
> at for Apache projects with Java components which all happened to do
> this (I'm sure there are others that are different, of course):
>
> https://hub.docker.com/r/_/tomcat/
> not-alpine: https://github.com/docker-library/tomcat/blob/
> 5ac222d258dc70c77bb3a9a4fab81ea286c9abd1/8.5/jre8/Dockerfile
> alpine: https://github.com/docker-library/tomcat/blob/
> 5ac222d258dc70c77bb3a9a4fab81ea286c9abd1/8.5/jre8-alpine/Dockerfile
>
> https://hub.docker.com/_/maven/
> not-alpine: https://github.com/carlossg/docker-maven/blob/
> 0490eff01e529b2d94789511b008d01a7b314953/jdk-8/Dockerfile
> alpine: https://github.com/carlossg/docker-maven/blob/
> 2357d3394f19730172ac9c7f4afe7cf052f36b4d/jdk-8/Dockerfile
>
> https://hub.docker.com/_/zookeeper/
> alpine: https://github.com/31z4/zookeeper-docker/blob/
> f12428ab7c6ea263ef037cf258129b83276c009c/3.4.10/Dockerfile
>
> At another try I got one thats doing something different:
>
> https://hub.docker.com/_/cassandra/
> https://github.com/docker-library/cassandra/blob/
> d83b850cd17bc9198876f8686197c730e29c7448/3.10/Dockerfile
>
> Here they seem to be using their own .deb files via
> http://www.apache.org/dist/cassandra/debian which actually redirects
> to http://dl.bintray.com/apache/cassandra/, a debian repo
> (https://bintray.com/apache/cassandra/debian) set up within the ASF
> org on bintray (https://bintray.com/apache)
>
> Robbie
>
> On 20 June 2017 at 11:05, Fraser Adams <fr...@blueyonder.co.uk>
> wrote:
> > Re: "it doesnt seem unusual to have a Dockerfile set up to pull the
> existing
> > binary release archive, verify its sigs, and extract+configure it in an
> > appropriate location."
> >
> > Yes, it's certainly not unusual, but my personal view is that it isn't
> great
> > practice.
> >
> > As I said in my earlier reply to Irina, IMHO there are far too many
> > instances of really bloaty Docker images containing far more than they
> need,
> > as well as unnecessarily making images larger than they need to be (which
> > isn't great if you are doing Continuous Deployment on a large system) it
> > also unnecessarily increases the attack surface. Now OK Qpid brokers are
> > probably long-lived services, so the first point might about minimising
> size
> > may apply less to them than say 12 Factor App business function services,
> > but as a general principle I tend to think that not enough thought is
> given
> > to the footprint of Docker images.
> >
> > I may have misunderstood, If the sentence I've quoted is referring to a
> > Dockerfile for a *build system*, which subsequently exports a zip
> containing
> > only that necessary to build (using a separate Dockerfile) a small,
> > versioned microcontainer based on a minimal distro like Alpine (or from
> > scratch) then that's fine, but having an image intended for use on a
> > production system doing that sort of thing doesn't seem appropriate to
> me.
> >
> > F.
> >
> >
> >
> > On 20/06/17 08:54, Lorenz Quack wrote:
> >>
> >> On Mon, 2017-06-19 at 13:16 +0100, Robbie Gemmell wrote:
> >>>
> >>> - To the comments around the Java broker, I don't think creating
> >>> packages for it is really necessary? From a quick look at some others
> >>> images it doesnt seem unusual to have a Dockerfile set up to pull the
> >>> existing binary release archive, verify its sigs, and
> >>> extract+configure it in an appropriate location.
> >>>
> >> Great. That would work for me.
> >> I just thought it would be good to have the entire Qpid project
> >> represented
> >> and to provide some choice at the same time.
> >>
> >> Kind regards,
> >> Lorenz
> >>
> >>> Robbie
> >>>
> >>> On 13 June 2017 at 15:13, Irina Boverman <ib...@redhat.com> wrote:
> >>>>
> >>>> Hi everyone,
> >>>>
> >>>> I would like to propose creating Docker images for Qpid components
> >>>> hosted
> >>>> in Docker Hub, updated upon component release and maintained by the
> >>>> project, and I would like to contribute to doing this.
> >>>>
> >>>> Availability of Qpid images will make it easier to consume/deploy Qpid
> >>>> components and promote Qpid visibility.
> >>>>
> >>>> We can maintain docker scripts creating these images from the base OS
> >>>> images and using Qpid installation methods consistent with the OS
> >>>> distribution. A possible naming convention might be
> >>>> qpid/<component>/<OS>.
> >>>> I registered the 'qpid' user on DockerHub to use if this seems
> >>>> reasonable.
> >>>> For example, we could create qpid/dispatch/<OS> image,
> >>>> qpid/<broker>/<OS>
> >>>> image, qpid/<client(s)>/<OS> image, etc. Initially I would look to
> >>>> support
> >>>> Fedora/CentOS latest images and Qpid components as RPMs for them, then
> >>>> aim
> >>>> to expand OS coverage for debian/Ubuntu/etc in the future.
> >>>>
> >>>> The goal would be to update Qpid images within a few days upon
> component
> >>>> release (either directly or indirectly using yum/dnf from public
> >>>> repositories). We could ask the Docker team to grant Qpid "official"
> >>>> status
> >>>> when images have been stabilized.
> >>>> --
> >>>> Regards, Irina.
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> >>> For additional commands, e-mail: dev-help@qpid.apache.org
> >>>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> >> For additional commands, e-mail: dev-help@qpid.apache.org
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> > For additional commands, e-mail: dev-help@qpid.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> For additional commands, e-mail: dev-help@qpid.apache.org
>
>
Re: Proposal to create Docker images for Qpid components.
Posted by Robbie Gemmell <ro...@gmail.com>.
It was talking about downloading the built Java broker binary release
tar.gz, verifying it, and doing something with it. It wasn't saying
anything in particular about the OS, except there is one and Java is
available somehow.
For example, some randomly selected 'docker official' images I looked
at for Apache projects with Java components which all happened to do
this (I'm sure there are others that are different, of course):
https://hub.docker.com/r/_/tomcat/
not-alpine: https://github.com/docker-library/tomcat/blob/5ac222d258dc70c77bb3a9a4fab81ea286c9abd1/8.5/jre8/Dockerfile
alpine: https://github.com/docker-library/tomcat/blob/5ac222d258dc70c77bb3a9a4fab81ea286c9abd1/8.5/jre8-alpine/Dockerfile
https://hub.docker.com/_/maven/
not-alpine: https://github.com/carlossg/docker-maven/blob/0490eff01e529b2d94789511b008d01a7b314953/jdk-8/Dockerfile
alpine: https://github.com/carlossg/docker-maven/blob/2357d3394f19730172ac9c7f4afe7cf052f36b4d/jdk-8/Dockerfile
https://hub.docker.com/_/zookeeper/
alpine: https://github.com/31z4/zookeeper-docker/blob/f12428ab7c6ea263ef037cf258129b83276c009c/3.4.10/Dockerfile
At another try I got one thats doing something different:
https://hub.docker.com/_/cassandra/
https://github.com/docker-library/cassandra/blob/d83b850cd17bc9198876f8686197c730e29c7448/3.10/Dockerfile
Here they seem to be using their own .deb files via
http://www.apache.org/dist/cassandra/debian which actually redirects
to http://dl.bintray.com/apache/cassandra/, a debian repo
(https://bintray.com/apache/cassandra/debian) set up within the ASF
org on bintray (https://bintray.com/apache)
Robbie
On 20 June 2017 at 11:05, Fraser Adams <fr...@blueyonder.co.uk> wrote:
> Re: "it doesnt seem unusual to have a Dockerfile set up to pull the existing
> binary release archive, verify its sigs, and extract+configure it in an
> appropriate location."
>
> Yes, it's certainly not unusual, but my personal view is that it isn't great
> practice.
>
> As I said in my earlier reply to Irina, IMHO there are far too many
> instances of really bloaty Docker images containing far more than they need,
> as well as unnecessarily making images larger than they need to be (which
> isn't great if you are doing Continuous Deployment on a large system) it
> also unnecessarily increases the attack surface. Now OK Qpid brokers are
> probably long-lived services, so the first point might about minimising size
> may apply less to them than say 12 Factor App business function services,
> but as a general principle I tend to think that not enough thought is given
> to the footprint of Docker images.
>
> I may have misunderstood, If the sentence I've quoted is referring to a
> Dockerfile for a *build system*, which subsequently exports a zip containing
> only that necessary to build (using a separate Dockerfile) a small,
> versioned microcontainer based on a minimal distro like Alpine (or from
> scratch) then that's fine, but having an image intended for use on a
> production system doing that sort of thing doesn't seem appropriate to me.
>
> F.
>
>
>
> On 20/06/17 08:54, Lorenz Quack wrote:
>>
>> On Mon, 2017-06-19 at 13:16 +0100, Robbie Gemmell wrote:
>>>
>>> - To the comments around the Java broker, I don't think creating
>>> packages for it is really necessary? From a quick look at some others
>>> images it doesnt seem unusual to have a Dockerfile set up to pull the
>>> existing binary release archive, verify its sigs, and
>>> extract+configure it in an appropriate location.
>>>
>> Great. That would work for me.
>> I just thought it would be good to have the entire Qpid project
>> represented
>> and to provide some choice at the same time.
>>
>> Kind regards,
>> Lorenz
>>
>>> Robbie
>>>
>>> On 13 June 2017 at 15:13, Irina Boverman <ib...@redhat.com> wrote:
>>>>
>>>> Hi everyone,
>>>>
>>>> I would like to propose creating Docker images for Qpid components
>>>> hosted
>>>> in Docker Hub, updated upon component release and maintained by the
>>>> project, and I would like to contribute to doing this.
>>>>
>>>> Availability of Qpid images will make it easier to consume/deploy Qpid
>>>> components and promote Qpid visibility.
>>>>
>>>> We can maintain docker scripts creating these images from the base OS
>>>> images and using Qpid installation methods consistent with the OS
>>>> distribution. A possible naming convention might be
>>>> qpid/<component>/<OS>.
>>>> I registered the 'qpid' user on DockerHub to use if this seems
>>>> reasonable.
>>>> For example, we could create qpid/dispatch/<OS> image,
>>>> qpid/<broker>/<OS>
>>>> image, qpid/<client(s)>/<OS> image, etc. Initially I would look to
>>>> support
>>>> Fedora/CentOS latest images and Qpid components as RPMs for them, then
>>>> aim
>>>> to expand OS coverage for debian/Ubuntu/etc in the future.
>>>>
>>>> The goal would be to update Qpid images within a few days upon component
>>>> release (either directly or indirectly using yum/dnf from public
>>>> repositories). We could ask the Docker team to grant Qpid "official"
>>>> status
>>>> when images have been stabilized.
>>>> --
>>>> Regards, Irina.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
>>> For additional commands, e-mail: dev-help@qpid.apache.org
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: dev-help@qpid.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> For additional commands, e-mail: dev-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Proposal to create Docker images for Qpid components.
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
Re: "it doesnt seem unusual to have a Dockerfile set up to pull the existing binary release archive, verify its sigs, and extract+configure it in an appropriate location."
Yes, it's certainly not unusual, but my personal view is that it isn't great practice.
As I said in my earlier reply to Irina, IMHO there are far too many instances of really bloaty Docker images containing far more than they need, as well as unnecessarily making images larger than they need to be (which isn't great if you are doing Continuous Deployment on a large system) it also unnecessarily increases the attack surface. Now OK Qpid brokers are probably long-lived services, so the first point might about minimising size may apply less to them than say 12 Factor App business function services, but as a general principle I tend to think that not enough thought is given to the footprint of Docker images.
I may have misunderstood, If the sentence I've quoted is referring to a Dockerfile for a *build system*, which subsequently exports a zip containing only that necessary to build (using a separate Dockerfile) a small, versioned microcontainer based on a minimal distro like Alpine (or from scratch) then that's fine, but having an image intended for use on a production system doing that sort of thing doesn't seem appropriate to me.
F.
On 20/06/17 08:54, Lorenz Quack wrote:
> On Mon, 2017-06-19 at 13:16 +0100, Robbie Gemmell wrote:
>> - To the comments around the Java broker, I don't think creating
>> packages for it is really necessary? From a quick look at some others
>> images it doesnt seem unusual to have a Dockerfile set up to pull the
>> existing binary release archive, verify its sigs, and
>> extract+configure it in an appropriate location.
>>
> Great. That would work for me.
> I just thought it would be good to have the entire Qpid project represented
> and to provide some choice at the same time.
>
> Kind regards,
> Lorenz
>
>> Robbie
>>
>> On 13 June 2017 at 15:13, Irina Boverman <ib...@redhat.com> wrote:
>>> Hi everyone,
>>>
>>> I would like to propose creating Docker images for Qpid components hosted
>>> in Docker Hub, updated upon component release and maintained by the
>>> project, and I would like to contribute to doing this.
>>>
>>> Availability of Qpid images will make it easier to consume/deploy Qpid
>>> components and promote Qpid visibility.
>>>
>>> We can maintain docker scripts creating these images from the base OS
>>> images and using Qpid installation methods consistent with the OS
>>> distribution. A possible naming convention might be qpid/<component>/<OS>.
>>> I registered the 'qpid' user on DockerHub to use if this seems reasonable.
>>> For example, we could create qpid/dispatch/<OS> image, qpid/<broker>/<OS>
>>> image, qpid/<client(s)>/<OS> image, etc. Initially I would look to support
>>> Fedora/CentOS latest images and Qpid components as RPMs for them, then aim
>>> to expand OS coverage for debian/Ubuntu/etc in the future.
>>>
>>> The goal would be to update Qpid images within a few days upon component
>>> release (either directly or indirectly using yum/dnf from public
>>> repositories). We could ask the Docker team to grant Qpid "official" status
>>> when images have been stabilized.
>>> --
>>> Regards, Irina.
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: dev-help@qpid.apache.org
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> For additional commands, e-mail: dev-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Proposal to create Docker images for Qpid components.
Posted by Lorenz Quack <qu...@gmail.com>.
On Mon, 2017-06-19 at 13:16 +0100, Robbie Gemmell wrote:
> - To the comments around the Java broker, I don't think creating
> packages for it is really necessary? From a quick look at some others
> images it doesnt seem unusual to have a Dockerfile set up to pull the
> existing binary release archive, verify its sigs, and
> extract+configure it in an appropriate location.
>
Great. That would work for me.
I just thought it would be good to have the entire Qpid project represented
and to provide some choice at the same time.
Kind regards,
Lorenz
> Robbie
>
> On 13 June 2017 at 15:13, Irina Boverman <ib...@redhat.com> wrote:
> >
> > Hi everyone,
> >
> > I would like to propose creating Docker images for Qpid components hosted
> > in Docker Hub, updated upon component release and maintained by the
> > project, and I would like to contribute to doing this.
> >
> > Availability of Qpid images will make it easier to consume/deploy Qpid
> > components and promote Qpid visibility.
> >
> > We can maintain docker scripts creating these images from the base OS
> > images and using Qpid installation methods consistent with the OS
> > distribution. A possible naming convention might be qpid/<component>/<OS>.
> > I registered the 'qpid' user on DockerHub to use if this seems reasonable.
> > For example, we could create qpid/dispatch/<OS> image, qpid/<broker>/<OS>
> > image, qpid/<client(s)>/<OS> image, etc. Initially I would look to support
> > Fedora/CentOS latest images and Qpid components as RPMs for them, then aim
> > to expand OS coverage for debian/Ubuntu/etc in the future.
> >
> > The goal would be to update Qpid images within a few days upon component
> > release (either directly or indirectly using yum/dnf from public
> > repositories). We could ask the Docker team to grant Qpid "official" status
> > when images have been stabilized.
> > --
> > Regards, Irina.
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> For additional commands, e-mail: dev-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Proposal to create Docker images for Qpid components.
Posted by Robbie Gemmell <ro...@gmail.com>.
I think docker images would be good to have.
Rather than reply to lots of little bits from various different mails,
I'm just going to gather some general comments/questions here based on
the previous mails.
- If I understand correctly Irina's proposal is essentially to create
create RPMs/DEBs (for certain components at least, might not be as
useful/necessary to others, see later) and then create docker images
using them (configure yum/dnf/apt repo, call install?). To Jakub's
point around doing things such as nightly RPMs, that could form part
of it however we still wouldn't especially promote them to end users,
much like the maven repo snapshots arent, as such interim builds are
only meant to be known to folks tracking/partipating in development as
we are only allowed to direct regular everyday users at project
releases.
- One thing I think worth some thought/discussion is versioning, e.g .
what would the docker tags offered be. It looks like some projects do
precise full-version tags to a specific 'major.minor.patch' release,
as well as more general 'minor version' and 'major version' tags
netting the latest release in a given stream. There might be some
related behaviour there to decide on, e.g if using the OS package
systems, would running an update result in moving to a newer component
release in the stream. For some of those tag formats it could, while
for others it should not.
- If we do start offering images based on packages, then those
packages and the related images essentially become 'convenience
binaries' for the release that get created+tested by the project as
part of the components release process. For bug fixes and
functionality updates to be usable in images, there will need to be
matching component releases to allow updating their packages and in
turn images. I think this points to the project doing more [patch]
releases for some of our components (i.e. ones we only do source
release archives for), which I think would also be a very good thing.
- Its true that having multiple base OS options would take
maintenance. Only having one option on the other hand probably
involves some decisions as to which (for a given component? Perhaps
they might not all be the same?). I think this also probably gets into
considering what people are looking for when they use a docker image.
In the case of a server such as a broker some people might just want
the port(s) listening and doing stuff to use/try-out a particular
server and dont care what OS etc is under it, while others might care
that that its on a particular OS variant (e.g one they use for all
their other bits), and others still might only care that its as small
an image as possible (somewhat combining the previous behaviours to an
extent).
- To the comments around the Java broker, I don't think creating
packages for it is really necessary? From a quick look at some others
images it doesnt seem unusual to have a Dockerfile set up to pull the
existing binary release archive, verify its sigs, and
extract+configure it in an appropriate location.
Robbie
On 13 June 2017 at 15:13, Irina Boverman <ib...@redhat.com> wrote:
> Hi everyone,
>
> I would like to propose creating Docker images for Qpid components hosted
> in Docker Hub, updated upon component release and maintained by the
> project, and I would like to contribute to doing this.
>
> Availability of Qpid images will make it easier to consume/deploy Qpid
> components and promote Qpid visibility.
>
> We can maintain docker scripts creating these images from the base OS
> images and using Qpid installation methods consistent with the OS
> distribution. A possible naming convention might be qpid/<component>/<OS>.
> I registered the 'qpid' user on DockerHub to use if this seems reasonable.
> For example, we could create qpid/dispatch/<OS> image, qpid/<broker>/<OS>
> image, qpid/<client(s)>/<OS> image, etc. Initially I would look to support
> Fedora/CentOS latest images and Qpid components as RPMs for them, then aim
> to expand OS coverage for debian/Ubuntu/etc in the future.
>
> The goal would be to update Qpid images within a few days upon component
> release (either directly or indirectly using yum/dnf from public
> repositories). We could ask the Docker team to grant Qpid "official" status
> when images have been stabilized.
> --
> Regards, Irina.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Proposal to create Docker images for Qpid components.
Posted by Jakub Scholz <ja...@scholz.cz>.
Definitely a good idea.
I have my own images for C++ broker (https://hub.docker.com/r/
scholzj/qpid-cpp/) / Dispatch (https://hub.docker.com/r/
scholzj/qpid-dispatch/) / GUI for the C++ broker (
https://hub.docker.com/r/scholzj/qpid-cpp-gui/) and since I built them for
the first time I probably never used anything else. So having something
official and centrally maintained would be great.
What I'm curios about is what would you (and of course also others) expect
in terms of how to configure the images - what env. variables should be
supported etc.
Getting it to the official docker library would be great. What I'm not sure
about is whether we really need the different OS versions. What would be
the benefit of maintaining them?
Regards
Jakub
On Tue, Jun 13, 2017 at 4:13 PM, Irina Boverman <ib...@redhat.com> wrote:
> Hi everyone,
>
> I would like to propose creating Docker images for Qpid components hosted
> in Docker Hub, updated upon component release and maintained by the
> project, and I would like to contribute to doing this.
>
> Availability of Qpid images will make it easier to consume/deploy Qpid
> components and promote Qpid visibility.
>
> We can maintain docker scripts creating these images from the base OS
> images and using Qpid installation methods consistent with the OS
> distribution. A possible naming convention might be qpid/<component>/<OS>.
> I registered the 'qpid' user on DockerHub to use if this seems reasonable.
> For example, we could create qpid/dispatch/<OS> image, qpid/<broker>/<OS>
> image, qpid/<client(s)>/<OS> image, etc. Initially I would look to support
> Fedora/CentOS latest images and Qpid components as RPMs for them, then aim
> to expand OS coverage for debian/Ubuntu/etc in the future.
>
> The goal would be to update Qpid images within a few days upon component
> release (either directly or indirectly using yum/dnf from public
> repositories). We could ask the Docker team to grant Qpid "official" status
> when images have been stabilized.
> --
> Regards, Irina.
>
RE: Proposal to create Docker images for Qpid components.
Posted by Steve Huston <sh...@riverace.com>.
+1
I think it's a great idea to make Docker images available.
Not sure it makes sense to package client(s) as images, but overall I think the idea would help make Qpid more easily accessible to users and help ease deployment.
-Steve
> -----Original Message-----
> From: Irina Boverman [mailto:iboverma@redhat.com]
> Sent: Tuesday, June 13, 2017 10:13 AM
> To: qpid <de...@qpid.apache.org>
> Subject: Proposal to create Docker images for Qpid components.
>
> Hi everyone,
>
> I would like to propose creating Docker images for Qpid components hosted
> in Docker Hub, updated upon component release and maintained by the
> project, and I would like to contribute to doing this.
>
> Availability of Qpid images will make it easier to consume/deploy Qpid
> components and promote Qpid visibility.
>
> We can maintain docker scripts creating these images from the base OS
> images and using Qpid installation methods consistent with the OS
> distribution. A possible naming convention might be
> qpid/<component>/<OS>.
> I registered the 'qpid' user on DockerHub to use if this seems reasonable.
> For example, we could create qpid/dispatch/<OS> image,
> qpid/<broker>/<OS> image, qpid/<client(s)>/<OS> image, etc. Initially I
> would look to support Fedora/CentOS latest images and Qpid components as
> RPMs for them, then aim to expand OS coverage for debian/Ubuntu/etc in
> the future.
>
> The goal would be to update Qpid images within a few days upon component
> release (either directly or indirectly using yum/dnf from public repositories).
> We could ask the Docker team to grant Qpid "official" status when images
> have been stabilized.
> --
> Regards, Irina.
Re: Proposal to create Docker images for Qpid components.
Posted by Lorenz Quack <qu...@gmail.com>.
Hi Irina,
Is there anything we could do to help/make it easier to package the java broker?
Are there any obstacles that make it hard to package it?
Kind regards,
Lorenz
On Tue, 2017-06-13 at 13:56 -0400, Irina Boverman wrote:
> As many as I can. Currently we do not package java broker as an RPM for
> Fedora/CentOS, nor do we have it packaged for Ubuntu.
>
>
> On Tue, Jun 13, 2017 at 12:03 PM, Lorenz Quack <qu...@gmail.com> wrote:
> > Hello Irina,
> >
> > Sounds like a good idea.
> >
> > Do you plan on providing docker images of all Qpid components?
> > Would be good to have for example the cpp and the java broker available to
> > easily evaluate and compare them.
> >
> > Kind regards,
> > Lorenz
> >
> >
> > On Tue, 2017-06-13 at 10:13 -0400, Irina Boverman wrote:
> > > Hi everyone,
> > >
> > > I would like to propose creating Docker images for Qpid components hosted
> > > in Docker Hub, updated upon component release and maintained by the
> > > project, and I would like to contribute to doing this.
> > >
> > > Availability of Qpid images will make it easier to consume/deploy Qpid
> > > components and promote Qpid visibility.
> > >
> > > We can maintain docker scripts creating these images from the base OS
> > > images and using Qpid installation methods consistent with the OS
> > > distribution. A possible naming convention might be qpid/<component>/<OS>.
> > > I registered the 'qpid' user on DockerHub to use if this seems reasonable.
> > > For example, we could create qpid/dispatch/<OS> image, qpid/<broker>/<OS>
> > > image, qpid/<client(s)>/<OS> image, etc. Initially I would look to support
> > > Fedora/CentOS latest images and Qpid components as RPMs for them, then aim
> > > to expand OS coverage for debian/Ubuntu/etc in the future.
> > >
> > > The goal would be to update Qpid images within a few days upon component
> > > release (either directly or indirectly using yum/dnf from public
> > > repositories). We could ask the Docker team to grant Qpid "official" status
> > > when images have been stabilized.
> > > --
> > > Regards, Irina.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> > For additional commands, e-mail: dev-help@qpid.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Proposal to create Docker images for Qpid components.
Posted by Irina Boverman <ib...@redhat.com>.
As many as I can. Currently we do not package java broker as an RPM for
Fedora/CentOS, nor do we have it packaged for Ubuntu.
On Tue, Jun 13, 2017 at 12:03 PM, Lorenz Quack <qu...@gmail.com>
wrote:
> Hello Irina,
>
> Sounds like a good idea.
>
> Do you plan on providing docker images of all Qpid components?
> Would be good to have for example the cpp and the java broker available to
> easily evaluate and compare them.
>
> Kind regards,
> Lorenz
>
>
> On Tue, 2017-06-13 at 10:13 -0400, Irina Boverman wrote:
> > Hi everyone,
> >
> > I would like to propose creating Docker images for Qpid components hosted
> > in Docker Hub, updated upon component release and maintained by the
> > project, and I would like to contribute to doing this.
> >
> > Availability of Qpid images will make it easier to consume/deploy Qpid
> > components and promote Qpid visibility.
> >
> > We can maintain docker scripts creating these images from the base OS
> > images and using Qpid installation methods consistent with the OS
> > distribution. A possible naming convention might be
> qpid/<component>/<OS>.
> > I registered the 'qpid' user on DockerHub to use if this seems
> reasonable.
> > For example, we could create qpid/dispatch/<OS> image, qpid/<broker>/<OS>
> > image, qpid/<client(s)>/<OS> image, etc. Initially I would look to
> support
> > Fedora/CentOS latest images and Qpid components as RPMs for them, then
> aim
> > to expand OS coverage for debian/Ubuntu/etc in the future.
> >
> > The goal would be to update Qpid images within a few days upon component
> > release (either directly or indirectly using yum/dnf from public
> > repositories). We could ask the Docker team to grant Qpid "official"
> status
> > when images have been stabilized.
> > --
> > Regards, Irina.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> For additional commands, e-mail: dev-help@qpid.apache.org
>
>
Re: Proposal to create Docker images for Qpid components.
Posted by Lorenz Quack <qu...@gmail.com>.
Hello Irina,
Sounds like a good idea.
Do you plan on providing docker images of all Qpid components?
Would be good to have for example the cpp and the java broker available to
easily evaluate and compare them.
Kind regards,
Lorenz
On Tue, 2017-06-13 at 10:13 -0400, Irina Boverman wrote:
> Hi everyone,
>
> I would like to propose creating Docker images for Qpid components hosted
> in Docker Hub, updated upon component release and maintained by the
> project, and I would like to contribute to doing this.
>
> Availability of Qpid images will make it easier to consume/deploy Qpid
> components and promote Qpid visibility.
>
> We can maintain docker scripts creating these images from the base OS
> images and using Qpid installation methods consistent with the OS
> distribution. A possible naming convention might be qpid/<component>/<OS>.
> I registered the 'qpid' user on DockerHub to use if this seems reasonable.
> For example, we could create qpid/dispatch/<OS> image, qpid/<broker>/<OS>
> image, qpid/<client(s)>/<OS> image, etc. Initially I would look to support
> Fedora/CentOS latest images and Qpid components as RPMs for them, then aim
> to expand OS coverage for debian/Ubuntu/etc in the future.
>
> The goal would be to update Qpid images within a few days upon component
> release (either directly or indirectly using yum/dnf from public
> repositories). We could ask the Docker team to grant Qpid "official" status
> when images have been stabilized.
> --
> Regards, Irina.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org