You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by David Morávek <dm...@apache.org> on 2021/12/22 13:42:36 UTC
[DISCUSS] Slimmed down docker images.
Hi,
I did some quick prototyping on the slimmed down docker images, and I was
able to cut the docker image size by ~40% with a minimum effort [1] (using
a multi-stage build + trimming examples / opt + using slimmed down JRE
image).
I think this might be a low hanging fruit for reducing MTTR in some
scenarios, but I'd like to hear other opinions on the topic.
Pushing this forward would require us to release twice as many images as we
do now (the current images + slimmed down versions).
Using /opt dependencies + /examples, would look something like this from an
user perspective:
FROM flink:1.14.0-scala_2.12-slim
COPY --from=flink:1.14.0-scala_2.12
/opt/flink/opt/flink-s3-fs-presto-1.15-SNAPSHOT.jar /opt/flink/plugins/s3
COPY --from=flink:1.14.0-scala_2.12
/opt/flink/examples/streaming/TopSpeedWindowing.jar /opt/flink/usrlib
Size of the 1.15 images (java 11):
~/Workspace/apache/flink-docker> docker images | grep flink | grep 1.15
flink 1.15
e96a3a3eaab2 15 minutes ago 702MB
flink 1.15-slim
e417b7665522 17 minutes ago 427MB
Do you see a benefits of this effort? Should image size / distribution size
be a concern with the modern deployments [2] ?
[1]
https://github.com/dmvk/flink-docker/commit/f866b3e57eacd0e6534b80fd0a1618cb30bbb36a
[2]
https://cloud.google.com/blog/products/containers-kubernetes/kubernetes-best-practices-how-and-why-to-build-small-container-images
Best,
D.
Re: [DISCUSS] Slimmed down docker images.
Posted by Chesnay Schepler <ch...@apache.org>.
There could be a problem with the fat image actually depending on the
slim version; I'm not sure if the official-images repo supports that.
We should however be able to generate 2 separate standalone dockerfiles.
On 23/12/2021 11:16, Till Rohrmann wrote:
> Hi David,
>
> Thanks for starting this discussion. I like the idea of providing smaller
> images that can be used by more advanced users that don't need everything.
> Having smaller image sizes can be really helpful when having to pull the
> image (with your changes this time should roughly be decreased by a factor
> of 2) in case of failure recovery or rescaling events.
>
> Concerning the added release complexity, can we automate the process of
> generating the slim and full images? If this is the case, then I don't see
> a big problem in having to release multiple images.
>
> Cheers,
> Till
>
> On Wed, Dec 22, 2021 at 2:43 PM David Morávek <dm...@apache.org> wrote:
>
>> Hi,
>>
>> I did some quick prototyping on the slimmed down docker images, and I was
>> able to cut the docker image size by ~40% with a minimum effort [1] (using
>> a multi-stage build + trimming examples / opt + using slimmed down JRE
>> image).
>>
>> I think this might be a low hanging fruit for reducing MTTR in some
>> scenarios, but I'd like to hear other opinions on the topic.
>>
>> Pushing this forward would require us to release twice as many images as we
>> do now (the current images + slimmed down versions).
>>
>> Using /opt dependencies + /examples, would look something like this from an
>> user perspective:
>>
>> FROM flink:1.14.0-scala_2.12-slim
>> COPY --from=flink:1.14.0-scala_2.12
>> /opt/flink/opt/flink-s3-fs-presto-1.15-SNAPSHOT.jar /opt/flink/plugins/s3
>> COPY --from=flink:1.14.0-scala_2.12
>> /opt/flink/examples/streaming/TopSpeedWindowing.jar /opt/flink/usrlib
>>
>> Size of the 1.15 images (java 11):
>>
>> ~/Workspace/apache/flink-docker> docker images | grep flink | grep 1.15
>> flink 1.15
>> e96a3a3eaab2 15 minutes ago 702MB
>> flink 1.15-slim
>> e417b7665522 17 minutes ago 427MB
>>
>>
>> Do you see a benefits of this effort? Should image size / distribution size
>> be a concern with the modern deployments [2] ?
>>
>> [1]
>>
>> https://github.com/dmvk/flink-docker/commit/f866b3e57eacd0e6534b80fd0a1618cb30bbb36a
>> [2]
>>
>> https://cloud.google.com/blog/products/containers-kubernetes/kubernetes-best-practices-how-and-why-to-build-small-container-images
>>
>> Best,
>> D.
>>
Re: [DISCUSS] Slimmed down docker images.
Posted by Till Rohrmann <tr...@apache.org>.
Hi David,
Thanks for starting this discussion. I like the idea of providing smaller
images that can be used by more advanced users that don't need everything.
Having smaller image sizes can be really helpful when having to pull the
image (with your changes this time should roughly be decreased by a factor
of 2) in case of failure recovery or rescaling events.
Concerning the added release complexity, can we automate the process of
generating the slim and full images? If this is the case, then I don't see
a big problem in having to release multiple images.
Cheers,
Till
On Wed, Dec 22, 2021 at 2:43 PM David Morávek <dm...@apache.org> wrote:
> Hi,
>
> I did some quick prototyping on the slimmed down docker images, and I was
> able to cut the docker image size by ~40% with a minimum effort [1] (using
> a multi-stage build + trimming examples / opt + using slimmed down JRE
> image).
>
> I think this might be a low hanging fruit for reducing MTTR in some
> scenarios, but I'd like to hear other opinions on the topic.
>
> Pushing this forward would require us to release twice as many images as we
> do now (the current images + slimmed down versions).
>
> Using /opt dependencies + /examples, would look something like this from an
> user perspective:
>
> FROM flink:1.14.0-scala_2.12-slim
> COPY --from=flink:1.14.0-scala_2.12
> /opt/flink/opt/flink-s3-fs-presto-1.15-SNAPSHOT.jar /opt/flink/plugins/s3
> COPY --from=flink:1.14.0-scala_2.12
> /opt/flink/examples/streaming/TopSpeedWindowing.jar /opt/flink/usrlib
>
> Size of the 1.15 images (java 11):
>
> ~/Workspace/apache/flink-docker> docker images | grep flink | grep 1.15
> flink 1.15
> e96a3a3eaab2 15 minutes ago 702MB
> flink 1.15-slim
> e417b7665522 17 minutes ago 427MB
>
>
> Do you see a benefits of this effort? Should image size / distribution size
> be a concern with the modern deployments [2] ?
>
> [1]
>
> https://github.com/dmvk/flink-docker/commit/f866b3e57eacd0e6534b80fd0a1618cb30bbb36a
> [2]
>
> https://cloud.google.com/blog/products/containers-kubernetes/kubernetes-best-practices-how-and-why-to-build-small-container-images
>
> Best,
> D.
>