Revisiting MINIFI-175; minifi-cpp Alpine Dockerfile

[Andrew Christianson <an...@nextcentury.com>]

Have we considered porting the current Dockerfile from Ubuntu to Alpine? The ubuntu image, especially with all the build-time deps left in the final image, is quite bloated. This runs against the design intent of minimal footprint in minifi-cpp. Further, the new multi-stage Docker builds would allow us to isolate the build environment in its own stage and allow only runtime deps to exist in the final image. Overall it would be a much smaller image.

Reviewing the JIRA activity for MINIFI-175, it seems the decision to not use alpine was originally due to the desire to encapsulate the build of minifi inside the build of the docker image, which for some reason wasn't compatible with using alpine. On initial testing, it seems alpine's musl libc has issues with spdlog. Since it may involve some effort and decision making with regard to which dependencies are used, this is a decision probably best made by the community.

Thoughts?

Re: Revisiting MINIFI-175; minifi-cpp Alpine Dockerfile

[Andrew Christianson <an...@nextcentury.com>]

It might be feasible to keep the build as-is (ubuntu-based), create a second stage based on alpine or busybox, then copy the minifi dist to the runtime. It would then be a question of library compatibility. We could either install the runtime deps (hopefully they are named and versioned in a compatible way) with apk add or copy the *.so from the build environment. At that point, it would be a matter of libc compatibility. Either musl glibc runtime works, or we install the runtime glibc for alpine [1] or the base image for it [2]. If we went the route of just copying out the linked *.so files, then a busybox base would be feasible and possibly smaller.

Is anyone aware of any other system deps minifi-cpp might have other than the linked shared libraries & glibc?

[1] https://github.com/sgerrand/alpine-pkg-glibc
[2] https://github.com/frol/docker-alpine-glibc
________________________________________
From: Jeremy Dyer <jd...@gmail.com>
Sent: Wednesday, April 26, 2017 11:36:17 AM
To: dev@nifi.apache.org
Subject: Re: Revisiting MINIFI-175; minifi-cpp Alpine Dockerfile

I'm all about smaller and alpine. To Aldrin's point I think we just wanted something out there at first and fully expected to optimize later

Sent from my iPhone

> On Apr 26, 2017, at 11:29 AM, Bryan Rosander <br...@apache.org> wrote:
>
> I'm definitely in favor of smaller images, especially given the nature of
> the subproject.  I think the MiNiFi Java image may be able to benefit from
> Alpine as well.
>
> Thanks,
> Bryan
>
>> On Wed, Apr 26, 2017 at 11:09 AM, Aldrin Piri <al...@gmail.com> wrote:
>>
>> Don't think there would be any objections to going smaller with Alpine and
>> certainly makes a lot of sense, but I believe initial efforts were
>> problematic and there was a general desire to get a foundation going.  The
>> biggest issue was needing to navigate cross compilation of a given host to
>> the Alpine format which was something our processes are not quite up to at
>> this juncture.  The integrated dev setup in the image avoided having to
>> tackle this and allowed building of the image on any system where Docker
>> could run whether natively or via one of the Docker for $OS setups.
>>
>> Hadn't actually been aware of the multi-stage docker builds before this
>> thread, but certainly seems like a great application after looking through
>> the associated PRs and docs and could be a more immediate win for the
>> particular case above.
>>
>> We could and should certainly strive to make components pluggable, in this
>> case logging, to support such cases where things may be problematic given a
>> target environment.  I don't think I'd be in favor of removing spdlog at
>> this point but could apply similar approaches used elsewhere in the
>> codebase to give us a little more configurability to use an alternate
>> implementation.
>>
>> Would be in favor of opening up an issue to optimize our Docker build and
>> certainly target Alpine with some of the new build magic.  By the time we
>> have it implemented, the multi-stage builds should likely be sufficiently
>> available.
>>
>> On Wed, Apr 26, 2017 at 10:19 AM, Andrew Christianson <
>> andrew.christianson@nextcentury.com> wrote:
>>
>>> Have we considered porting the current Dockerfile from Ubuntu to Alpine?
>>> The ubuntu image, especially with all the build-time deps left in the
>> final
>>> image, is quite bloated. This runs against the design intent of minimal
>>> footprint in minifi-cpp. Further, the new multi-stage Docker builds would
>>> allow us to isolate the build environment in its own stage and allow only
>>> runtime deps to exist in the final image. Overall it would be a much
>>> smaller image.
>>>
>>> Reviewing the JIRA activity for MINIFI-175, it seems the decision to not
>>> use alpine was originally due to the desire to encapsulate the build of
>>> minifi inside the build of the docker image, which for some reason wasn't
>>> compatible with using alpine. On initial testing, it seems alpine's musl
>>> libc has issues with spdlog. Since it may involve some effort and
>> decision
>>> making with regard to which dependencies are used, this is a decision
>>> probably best made by the community.
>>>
>>> Thoughts?
>>

Re: Revisiting MINIFI-175; minifi-cpp Alpine Dockerfile

[Jeremy Dyer <jd...@gmail.com>]

I'm all about smaller and alpine. To Aldrin's point I think we just wanted something out there at first and fully expected to optimize later

Sent from my iPhone

> On Apr 26, 2017, at 11:29 AM, Bryan Rosander <br...@apache.org> wrote:
> 
> I'm definitely in favor of smaller images, especially given the nature of
> the subproject.  I think the MiNiFi Java image may be able to benefit from
> Alpine as well.
> 
> Thanks,
> Bryan
> 
>> On Wed, Apr 26, 2017 at 11:09 AM, Aldrin Piri <al...@gmail.com> wrote:
>> 
>> Don't think there would be any objections to going smaller with Alpine and
>> certainly makes a lot of sense, but I believe initial efforts were
>> problematic and there was a general desire to get a foundation going.  The
>> biggest issue was needing to navigate cross compilation of a given host to
>> the Alpine format which was something our processes are not quite up to at
>> this juncture.  The integrated dev setup in the image avoided having to
>> tackle this and allowed building of the image on any system where Docker
>> could run whether natively or via one of the Docker for $OS setups.
>> 
>> Hadn't actually been aware of the multi-stage docker builds before this
>> thread, but certainly seems like a great application after looking through
>> the associated PRs and docs and could be a more immediate win for the
>> particular case above.
>> 
>> We could and should certainly strive to make components pluggable, in this
>> case logging, to support such cases where things may be problematic given a
>> target environment.  I don't think I'd be in favor of removing spdlog at
>> this point but could apply similar approaches used elsewhere in the
>> codebase to give us a little more configurability to use an alternate
>> implementation.
>> 
>> Would be in favor of opening up an issue to optimize our Docker build and
>> certainly target Alpine with some of the new build magic.  By the time we
>> have it implemented, the multi-stage builds should likely be sufficiently
>> available.
>> 
>> On Wed, Apr 26, 2017 at 10:19 AM, Andrew Christianson <
>> andrew.christianson@nextcentury.com> wrote:
>> 
>>> Have we considered porting the current Dockerfile from Ubuntu to Alpine?
>>> The ubuntu image, especially with all the build-time deps left in the
>> final
>>> image, is quite bloated. This runs against the design intent of minimal
>>> footprint in minifi-cpp. Further, the new multi-stage Docker builds would
>>> allow us to isolate the build environment in its own stage and allow only
>>> runtime deps to exist in the final image. Overall it would be a much
>>> smaller image.
>>> 
>>> Reviewing the JIRA activity for MINIFI-175, it seems the decision to not
>>> use alpine was originally due to the desire to encapsulate the build of
>>> minifi inside the build of the docker image, which for some reason wasn't
>>> compatible with using alpine. On initial testing, it seems alpine's musl
>>> libc has issues with spdlog. Since it may involve some effort and
>> decision
>>> making with regard to which dependencies are used, this is a decision
>>> probably best made by the community.
>>> 
>>> Thoughts?
>> 

Re: Revisiting MINIFI-175; minifi-cpp Alpine Dockerfile

[Bryan Rosander <br...@apache.org>]

I'm definitely in favor of smaller images, especially given the nature of
the subproject.  I think the MiNiFi Java image may be able to benefit from
Alpine as well.

Thanks,
Bryan

On Wed, Apr 26, 2017 at 11:09 AM, Aldrin Piri <al...@gmail.com> wrote:

> Don't think there would be any objections to going smaller with Alpine and
> certainly makes a lot of sense, but I believe initial efforts were
> problematic and there was a general desire to get a foundation going.  The
> biggest issue was needing to navigate cross compilation of a given host to
> the Alpine format which was something our processes are not quite up to at
> this juncture.  The integrated dev setup in the image avoided having to
> tackle this and allowed building of the image on any system where Docker
> could run whether natively or via one of the Docker for $OS setups.
>
> Hadn't actually been aware of the multi-stage docker builds before this
> thread, but certainly seems like a great application after looking through
> the associated PRs and docs and could be a more immediate win for the
> particular case above.
>
> We could and should certainly strive to make components pluggable, in this
> case logging, to support such cases where things may be problematic given a
> target environment.  I don't think I'd be in favor of removing spdlog at
> this point but could apply similar approaches used elsewhere in the
> codebase to give us a little more configurability to use an alternate
> implementation.
>
> Would be in favor of opening up an issue to optimize our Docker build and
> certainly target Alpine with some of the new build magic.  By the time we
> have it implemented, the multi-stage builds should likely be sufficiently
> available.
>
> On Wed, Apr 26, 2017 at 10:19 AM, Andrew Christianson <
> andrew.christianson@nextcentury.com> wrote:
>
> > Have we considered porting the current Dockerfile from Ubuntu to Alpine?
> > The ubuntu image, especially with all the build-time deps left in the
> final
> > image, is quite bloated. This runs against the design intent of minimal
> > footprint in minifi-cpp. Further, the new multi-stage Docker builds would
> > allow us to isolate the build environment in its own stage and allow only
> > runtime deps to exist in the final image. Overall it would be a much
> > smaller image.
> >
> > Reviewing the JIRA activity for MINIFI-175, it seems the decision to not
> > use alpine was originally due to the desire to encapsulate the build of
> > minifi inside the build of the docker image, which for some reason wasn't
> > compatible with using alpine. On initial testing, it seems alpine's musl
> > libc has issues with spdlog. Since it may involve some effort and
> decision
> > making with regard to which dependencies are used, this is a decision
> > probably best made by the community.
> >
> > Thoughts?
>

Re: Revisiting MINIFI-175; minifi-cpp Alpine Dockerfile

[Aldrin Piri <al...@gmail.com>]

Don't think there would be any objections to going smaller with Alpine and
certainly makes a lot of sense, but I believe initial efforts were
problematic and there was a general desire to get a foundation going.  The
biggest issue was needing to navigate cross compilation of a given host to
the Alpine format which was something our processes are not quite up to at
this juncture.  The integrated dev setup in the image avoided having to
tackle this and allowed building of the image on any system where Docker
could run whether natively or via one of the Docker for $OS setups.

Hadn't actually been aware of the multi-stage docker builds before this
thread, but certainly seems like a great application after looking through
the associated PRs and docs and could be a more immediate win for the
particular case above.

We could and should certainly strive to make components pluggable, in this
case logging, to support such cases where things may be problematic given a
target environment.  I don't think I'd be in favor of removing spdlog at
this point but could apply similar approaches used elsewhere in the
codebase to give us a little more configurability to use an alternate
implementation.

Would be in favor of opening up an issue to optimize our Docker build and
certainly target Alpine with some of the new build magic.  By the time we
have it implemented, the multi-stage builds should likely be sufficiently
available.

On Wed, Apr 26, 2017 at 10:19 AM, Andrew Christianson <
andrew.christianson@nextcentury.com> wrote:

> Have we considered porting the current Dockerfile from Ubuntu to Alpine?
> The ubuntu image, especially with all the build-time deps left in the final
> image, is quite bloated. This runs against the design intent of minimal
> footprint in minifi-cpp. Further, the new multi-stage Docker builds would
> allow us to isolate the build environment in its own stage and allow only
> runtime deps to exist in the final image. Overall it would be a much
> smaller image.
>
> Reviewing the JIRA activity for MINIFI-175, it seems the decision to not
> use alpine was originally due to the desire to encapsulate the build of
> minifi inside the build of the docker image, which for some reason wasn't
> compatible with using alpine. On initial testing, it seems alpine's musl
> libc has issues with spdlog. Since it may involve some effort and decision
> making with regard to which dependencies are used, this is a decision
> probably best made by the community.
>
> Thoughts?