You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Patrick Lucas <pa...@data-artisans.com> on 2017/04/27 10:21:31 UTC
Official Flink Docker images
As part of an ongoing effort to improve the experience of using Flink on
Docker, some work has been done over the last two months to publish
official Flink Docker images to Docker Hub. The goal in the short term is
to make running a simple Flink cluster (almost) as easy as running docker
run flink. In the long term, we would like these images to be good enough
to use in production directly, or as base images for use in an existing
Docker workflow.
Flink 1.2.1 has some fixes over the last few releases that make running it
on Docker nicer—and in some cases, possible. Notably, FLINK-2821
<https://issues.apache.org/jira/browse/FLINK-2821> allowed Dockerized Flink
to run across multiple hosts, and FLINK-4326
<https://issues.apache.org/jira/browse/FLINK-4326> added an option to run
Flink in the foreground, which is greatly preferred when running in Docker.
We (Ismaël Mejía and myself, with some discussion with Stephan Ewen)
decided it made sense to bring the actual Dockerfiles outside of the Apache
Flink git repo, primarily to conform with every other Apache project that
has official images, but also because these scripts logically exist
decoupled from any particular Flink version. They are still Apache-licensed
and maintained by the community.
Please reply here or on the relevant JIRA/GitHub issue if you have
questions or feedback.
Here's a squirrel in a container:
References:
- FLINK-3026: Publish the flink docker container to the docker registry
<https://issues.apache.org/jira/browse/FLINK-3026>
- Repo for the Dockerfiles and the scripts that generate them
<https://github.com/docker-flink/docker-flink>
- GitHub PR to add the official images to Docker Hub
<https://github.com/docker-library/official-images/pull/2895>
- Improvements to the quality of running Flink on Docker to be made in
future Flink releases:
- FLINK-6300: PID1 of docker images does not behave correctly
<https://issues.apache.org/jira/browse/FLINK-6300>
- FLINK-6369: Better support for overlay networks
<https://issues.apache.org/jira/browse/FLINK-6369>
Thanks to Ismaël Mejía, Jamie Grier, and Stephan Ewen for their
contributions.
--
Patrick Lucas
Re: Official Flink Docker images
Posted by Henry Saputra <he...@gmail.com>.
We already have a Dockerfile in our source repo as part of simple test:
https://github.com/apache/flink/tree/master/flink-contrib/docker-flink
but it was never automatically build by our build system AFAIK.
- Henry
On Tue, May 16, 2017 at 8:58 AM, Robert Metzger <rm...@apache.org> wrote:
> How hard would it be to integrate the docker images into the Flink release
> process?
>
> Ideally, we could provide something like a staging directory for the docker
> images, so that we can include them into the vote.
> Once the vote has passed, the images will be made public through docker hub
> and apache.
>
> On Tue, May 16, 2017 at 3:12 PM, Till Rohrmann <tr...@apache.org>
> wrote:
>
> > Great to hear Ismaël. This will make running Flink even easier :-)
> >
> > Cheers,
> > Till
> >
> > On Tue, May 16, 2017 at 1:38 PM, Ismaël Mejía <ie...@gmail.com> wrote:
> >
> > > As a follow up for this thread, the docker official images are out
> > > since last week ago so you please guys go ahead and try them. The blog
> > > post that presents them was published today.
> > >
> > > https://flink.apache.org/news/2017/05/16/official-docker-image.html
> > >
> > > On Sun, May 7, 2017 at 3:51 PM, Ismaël Mejía <ie...@gmail.com>
> wrote:
> > > > Sure we would do it. I will sync with Patrick to do this quickly.
> > > >
> > > > On May 5, 2017 3:45 PM, "Robert Metzger" <rm...@apache.org>
> wrote:
> > > >>
> > > >> Thanks a lot for your efforts!
> > > >>
> > > >> Maybe we can even put a small blog post on the Flink post once its
> > > >> available on docker hub.
> > > >>
> > > >> On Fri, Apr 28, 2017 at 11:00 AM, Ismaël Mejía <ie...@gmail.com>
> > > wrote:
> > > >>
> > > >> > Hello,
> > > >> >
> > > >> > I am absolutely happy to see that this is finally happening!
> > > >> > We have a really neat image right now and it is great that it will
> > be
> > > >> > soon so easy to use.
> > > >> >
> > > >> > One extra thing to mention is that Flink will have now two docker
> > > >> > images, one based on debian and the other one based on Alpine as
> > most
> > > >> > official java-based projects do.
> > > >> >
> > > >> > In the future we expect to improve the documentation on how to use
> > the
> > > >> > image with kubernetes and continue improving the actual
> > documentation
> > > >> > with docker. If anyone wants to join to also document something or
> > add
> > > >> > any improvement/feature you need, you are all welcome.
> > > >> >
> > > >> > Finally, I would also like to thank Maximilian Michels which
> > > >> > contributed and reviewed some of my early changes on the image.
> > > >> >
> > > >> > Regards,
> > > >> > Ismaël
> > > >> >
> > > >> > ps. We will 'announce' when the official images are available on
> > > >> > docker hub, so everyone can start to use them.
> > > >> >
> > > >> >
> > > >> >
> > > >> > On Thu, Apr 27, 2017 at 1:38 PM, Patrick Lucas
> > > >> > <pa...@data-artisans.com> wrote:
> > > >> > > I've been informed that images don't make it through the list!
> > > >> > >
> > > >> > > You can see the aforementioned squirrel here
> > > >> > > <https://activerain-store.s3.amazonaws.com/image_store/
> > > >> > uploads/8/7/6/3/9/ar12988558393678.JPG>
> > > >> > > .
> > > >> > >
> > > >> > > --
> > > >> > > Patrick Lucas
> > > >> > >
> > > >> > > On Thu, Apr 27, 2017 at 12:21 PM, Patrick Lucas <
> > > >> > patrick@data-artisans.com>
> > > >> > > wrote:
> > > >> > >
> > > >> > >> As part of an ongoing effort to improve the experience of using
> > > Flink
> > > >> > >> on
> > > >> > >> Docker, some work has been done over the last two months to
> > publish
> > > >> > >> official Flink Docker images to Docker Hub. The goal in the
> short
> > > >> > >> term
> > > >> > is
> > > >> > >> to make running a simple Flink cluster (almost) as easy as
> > running
> > > >> > docker
> > > >> > >> run flink. In the long term, we would like these images to be
> > good
> > > >> > enough
> > > >> > >> to use in production directly, or as base images for use in an
> > > >> > >> existing
> > > >> > >> Docker workflow.
> > > >> > >>
> > > >> > >> Flink 1.2.1 has some fixes over the last few releases that make
> > > >> > >> running
> > > >> > it
> > > >> > >> on Docker nicer—and in some cases, possible. Notably,
> FLINK-2821
> > > >> > >> <https://issues.apache.org/jira/browse/FLINK-2821> allowed
> > > Dockerized
> > > >> > >> Flink to run across multiple hosts, and FLINK-4326
> > > >> > >> <https://issues.apache.org/jira/browse/FLINK-4326> added an
> > > option to
> > > >> > run
> > > >> > >> Flink in the foreground, which is greatly preferred when
> running
> > in
> > > >> > Docker.
> > > >> > >>
> > > >> > >> We (Ismaël Mejía and myself, with some discussion with Stephan
> > > Ewen)
> > > >> > >> decided it made sense to bring the actual Dockerfiles outside
> of
> > > the
> > > >> > Apache
> > > >> > >> Flink git repo, primarily to conform with every other Apache
> > > project
> > > >> > that
> > > >> > >> has official images, but also because these scripts logically
> > exist
> > > >> > >> decoupled from any particular Flink version. They are still
> > > >> > Apache-licensed
> > > >> > >> and maintained by the community.
> > > >> > >>
> > > >> > >> Please reply here or on the relevant JIRA/GitHub issue if you
> > have
> > > >> > >> questions or feedback.
> > > >> > >>
> > > >> > >> Here's a squirrel in a container:
> > > >> > >>
> > > >> > >>
> > > >> > >>
> > > >> > >> References:
> > > >> > >>
> > > >> > >> - FLINK-3026: Publish the flink docker container to the
> docker
> > > >> > registry
> > > >> > >> <https://issues.apache.org/jira/browse/FLINK-3026>
> > > >> > >> - Repo for the Dockerfiles and the scripts that generate
> them
> > > >> > >> <https://github.com/docker-flink/docker-flink>
> > > >> > >> - GitHub PR to add the official images to Docker Hub
> > > >> > >> <https://github.com/docker-library/official-images/pull/
> 2895>
> > > >> > >> - Improvements to the quality of running Flink on Docker to
> be
> > > >> > >> made
> > > >> > in
> > > >> > >> future Flink releases:
> > > >> > >> - FLINK-6300: PID1 of docker images does not behave
> > correctly
> > > >> > >> <https://issues.apache.org/jira/browse/FLINK-6300>
> > > >> > >> - FLINK-6369: Better support for overlay networks
> > > >> > >> <https://issues.apache.org/jira/browse/FLINK-6369>
> > > >> > >>
> > > >> > >> Thanks to Ismaël Mejía, Jamie Grier, and Stephan Ewen for their
> > > >> > >> contributions.
> > > >> > >>
> > > >> > >> --
> > > >> > >> Patrick Lucas
> > > >> > >>
> > > >> >
> > >
> >
>
Re: Official Flink Docker images
Posted by Robert Metzger <rm...@apache.org>.
How hard would it be to integrate the docker images into the Flink release
process?
Ideally, we could provide something like a staging directory for the docker
images, so that we can include them into the vote.
Once the vote has passed, the images will be made public through docker hub
and apache.
On Tue, May 16, 2017 at 3:12 PM, Till Rohrmann <tr...@apache.org> wrote:
> Great to hear Ismaël. This will make running Flink even easier :-)
>
> Cheers,
> Till
>
> On Tue, May 16, 2017 at 1:38 PM, Ismaël Mejía <ie...@gmail.com> wrote:
>
> > As a follow up for this thread, the docker official images are out
> > since last week ago so you please guys go ahead and try them. The blog
> > post that presents them was published today.
> >
> > https://flink.apache.org/news/2017/05/16/official-docker-image.html
> >
> > On Sun, May 7, 2017 at 3:51 PM, Ismaël Mejía <ie...@gmail.com> wrote:
> > > Sure we would do it. I will sync with Patrick to do this quickly.
> > >
> > > On May 5, 2017 3:45 PM, "Robert Metzger" <rm...@apache.org> wrote:
> > >>
> > >> Thanks a lot for your efforts!
> > >>
> > >> Maybe we can even put a small blog post on the Flink post once its
> > >> available on docker hub.
> > >>
> > >> On Fri, Apr 28, 2017 at 11:00 AM, Ismaël Mejía <ie...@gmail.com>
> > wrote:
> > >>
> > >> > Hello,
> > >> >
> > >> > I am absolutely happy to see that this is finally happening!
> > >> > We have a really neat image right now and it is great that it will
> be
> > >> > soon so easy to use.
> > >> >
> > >> > One extra thing to mention is that Flink will have now two docker
> > >> > images, one based on debian and the other one based on Alpine as
> most
> > >> > official java-based projects do.
> > >> >
> > >> > In the future we expect to improve the documentation on how to use
> the
> > >> > image with kubernetes and continue improving the actual
> documentation
> > >> > with docker. If anyone wants to join to also document something or
> add
> > >> > any improvement/feature you need, you are all welcome.
> > >> >
> > >> > Finally, I would also like to thank Maximilian Michels which
> > >> > contributed and reviewed some of my early changes on the image.
> > >> >
> > >> > Regards,
> > >> > Ismaël
> > >> >
> > >> > ps. We will 'announce' when the official images are available on
> > >> > docker hub, so everyone can start to use them.
> > >> >
> > >> >
> > >> >
> > >> > On Thu, Apr 27, 2017 at 1:38 PM, Patrick Lucas
> > >> > <pa...@data-artisans.com> wrote:
> > >> > > I've been informed that images don't make it through the list!
> > >> > >
> > >> > > You can see the aforementioned squirrel here
> > >> > > <https://activerain-store.s3.amazonaws.com/image_store/
> > >> > uploads/8/7/6/3/9/ar12988558393678.JPG>
> > >> > > .
> > >> > >
> > >> > > --
> > >> > > Patrick Lucas
> > >> > >
> > >> > > On Thu, Apr 27, 2017 at 12:21 PM, Patrick Lucas <
> > >> > patrick@data-artisans.com>
> > >> > > wrote:
> > >> > >
> > >> > >> As part of an ongoing effort to improve the experience of using
> > Flink
> > >> > >> on
> > >> > >> Docker, some work has been done over the last two months to
> publish
> > >> > >> official Flink Docker images to Docker Hub. The goal in the short
> > >> > >> term
> > >> > is
> > >> > >> to make running a simple Flink cluster (almost) as easy as
> running
> > >> > docker
> > >> > >> run flink. In the long term, we would like these images to be
> good
> > >> > enough
> > >> > >> to use in production directly, or as base images for use in an
> > >> > >> existing
> > >> > >> Docker workflow.
> > >> > >>
> > >> > >> Flink 1.2.1 has some fixes over the last few releases that make
> > >> > >> running
> > >> > it
> > >> > >> on Docker nicer—and in some cases, possible. Notably, FLINK-2821
> > >> > >> <https://issues.apache.org/jira/browse/FLINK-2821> allowed
> > Dockerized
> > >> > >> Flink to run across multiple hosts, and FLINK-4326
> > >> > >> <https://issues.apache.org/jira/browse/FLINK-4326> added an
> > option to
> > >> > run
> > >> > >> Flink in the foreground, which is greatly preferred when running
> in
> > >> > Docker.
> > >> > >>
> > >> > >> We (Ismaël Mejía and myself, with some discussion with Stephan
> > Ewen)
> > >> > >> decided it made sense to bring the actual Dockerfiles outside of
> > the
> > >> > Apache
> > >> > >> Flink git repo, primarily to conform with every other Apache
> > project
> > >> > that
> > >> > >> has official images, but also because these scripts logically
> exist
> > >> > >> decoupled from any particular Flink version. They are still
> > >> > Apache-licensed
> > >> > >> and maintained by the community.
> > >> > >>
> > >> > >> Please reply here or on the relevant JIRA/GitHub issue if you
> have
> > >> > >> questions or feedback.
> > >> > >>
> > >> > >> Here's a squirrel in a container:
> > >> > >>
> > >> > >>
> > >> > >>
> > >> > >> References:
> > >> > >>
> > >> > >> - FLINK-3026: Publish the flink docker container to the docker
> > >> > registry
> > >> > >> <https://issues.apache.org/jira/browse/FLINK-3026>
> > >> > >> - Repo for the Dockerfiles and the scripts that generate them
> > >> > >> <https://github.com/docker-flink/docker-flink>
> > >> > >> - GitHub PR to add the official images to Docker Hub
> > >> > >> <https://github.com/docker-library/official-images/pull/2895>
> > >> > >> - Improvements to the quality of running Flink on Docker to be
> > >> > >> made
> > >> > in
> > >> > >> future Flink releases:
> > >> > >> - FLINK-6300: PID1 of docker images does not behave
> correctly
> > >> > >> <https://issues.apache.org/jira/browse/FLINK-6300>
> > >> > >> - FLINK-6369: Better support for overlay networks
> > >> > >> <https://issues.apache.org/jira/browse/FLINK-6369>
> > >> > >>
> > >> > >> Thanks to Ismaël Mejía, Jamie Grier, and Stephan Ewen for their
> > >> > >> contributions.
> > >> > >>
> > >> > >> --
> > >> > >> Patrick Lucas
> > >> > >>
> > >> >
> >
>
Re: Official Flink Docker images
Posted by Till Rohrmann <tr...@apache.org>.
Great to hear Ismaël. This will make running Flink even easier :-)
Cheers,
Till
On Tue, May 16, 2017 at 1:38 PM, Ismaël Mejía <ie...@gmail.com> wrote:
> As a follow up for this thread, the docker official images are out
> since last week ago so you please guys go ahead and try them. The blog
> post that presents them was published today.
>
> https://flink.apache.org/news/2017/05/16/official-docker-image.html
>
> On Sun, May 7, 2017 at 3:51 PM, Ismaël Mejía <ie...@gmail.com> wrote:
> > Sure we would do it. I will sync with Patrick to do this quickly.
> >
> > On May 5, 2017 3:45 PM, "Robert Metzger" <rm...@apache.org> wrote:
> >>
> >> Thanks a lot for your efforts!
> >>
> >> Maybe we can even put a small blog post on the Flink post once its
> >> available on docker hub.
> >>
> >> On Fri, Apr 28, 2017 at 11:00 AM, Ismaël Mejía <ie...@gmail.com>
> wrote:
> >>
> >> > Hello,
> >> >
> >> > I am absolutely happy to see that this is finally happening!
> >> > We have a really neat image right now and it is great that it will be
> >> > soon so easy to use.
> >> >
> >> > One extra thing to mention is that Flink will have now two docker
> >> > images, one based on debian and the other one based on Alpine as most
> >> > official java-based projects do.
> >> >
> >> > In the future we expect to improve the documentation on how to use the
> >> > image with kubernetes and continue improving the actual documentation
> >> > with docker. If anyone wants to join to also document something or add
> >> > any improvement/feature you need, you are all welcome.
> >> >
> >> > Finally, I would also like to thank Maximilian Michels which
> >> > contributed and reviewed some of my early changes on the image.
> >> >
> >> > Regards,
> >> > Ismaël
> >> >
> >> > ps. We will 'announce' when the official images are available on
> >> > docker hub, so everyone can start to use them.
> >> >
> >> >
> >> >
> >> > On Thu, Apr 27, 2017 at 1:38 PM, Patrick Lucas
> >> > <pa...@data-artisans.com> wrote:
> >> > > I've been informed that images don't make it through the list!
> >> > >
> >> > > You can see the aforementioned squirrel here
> >> > > <https://activerain-store.s3.amazonaws.com/image_store/
> >> > uploads/8/7/6/3/9/ar12988558393678.JPG>
> >> > > .
> >> > >
> >> > > --
> >> > > Patrick Lucas
> >> > >
> >> > > On Thu, Apr 27, 2017 at 12:21 PM, Patrick Lucas <
> >> > patrick@data-artisans.com>
> >> > > wrote:
> >> > >
> >> > >> As part of an ongoing effort to improve the experience of using
> Flink
> >> > >> on
> >> > >> Docker, some work has been done over the last two months to publish
> >> > >> official Flink Docker images to Docker Hub. The goal in the short
> >> > >> term
> >> > is
> >> > >> to make running a simple Flink cluster (almost) as easy as running
> >> > docker
> >> > >> run flink. In the long term, we would like these images to be good
> >> > enough
> >> > >> to use in production directly, or as base images for use in an
> >> > >> existing
> >> > >> Docker workflow.
> >> > >>
> >> > >> Flink 1.2.1 has some fixes over the last few releases that make
> >> > >> running
> >> > it
> >> > >> on Docker nicer—and in some cases, possible. Notably, FLINK-2821
> >> > >> <https://issues.apache.org/jira/browse/FLINK-2821> allowed
> Dockerized
> >> > >> Flink to run across multiple hosts, and FLINK-4326
> >> > >> <https://issues.apache.org/jira/browse/FLINK-4326> added an
> option to
> >> > run
> >> > >> Flink in the foreground, which is greatly preferred when running in
> >> > Docker.
> >> > >>
> >> > >> We (Ismaël Mejía and myself, with some discussion with Stephan
> Ewen)
> >> > >> decided it made sense to bring the actual Dockerfiles outside of
> the
> >> > Apache
> >> > >> Flink git repo, primarily to conform with every other Apache
> project
> >> > that
> >> > >> has official images, but also because these scripts logically exist
> >> > >> decoupled from any particular Flink version. They are still
> >> > Apache-licensed
> >> > >> and maintained by the community.
> >> > >>
> >> > >> Please reply here or on the relevant JIRA/GitHub issue if you have
> >> > >> questions or feedback.
> >> > >>
> >> > >> Here's a squirrel in a container:
> >> > >>
> >> > >>
> >> > >>
> >> > >> References:
> >> > >>
> >> > >> - FLINK-3026: Publish the flink docker container to the docker
> >> > registry
> >> > >> <https://issues.apache.org/jira/browse/FLINK-3026>
> >> > >> - Repo for the Dockerfiles and the scripts that generate them
> >> > >> <https://github.com/docker-flink/docker-flink>
> >> > >> - GitHub PR to add the official images to Docker Hub
> >> > >> <https://github.com/docker-library/official-images/pull/2895>
> >> > >> - Improvements to the quality of running Flink on Docker to be
> >> > >> made
> >> > in
> >> > >> future Flink releases:
> >> > >> - FLINK-6300: PID1 of docker images does not behave correctly
> >> > >> <https://issues.apache.org/jira/browse/FLINK-6300>
> >> > >> - FLINK-6369: Better support for overlay networks
> >> > >> <https://issues.apache.org/jira/browse/FLINK-6369>
> >> > >>
> >> > >> Thanks to Ismaël Mejía, Jamie Grier, and Stephan Ewen for their
> >> > >> contributions.
> >> > >>
> >> > >> --
> >> > >> Patrick Lucas
> >> > >>
> >> >
>
Re: Official Flink Docker images
Posted by Ismaël Mejía <ie...@gmail.com>.
As a follow up for this thread, the docker official images are out
since last week ago so you please guys go ahead and try them. The blog
post that presents them was published today.
https://flink.apache.org/news/2017/05/16/official-docker-image.html
On Sun, May 7, 2017 at 3:51 PM, Ismaël Mejía <ie...@gmail.com> wrote:
> Sure we would do it. I will sync with Patrick to do this quickly.
>
> On May 5, 2017 3:45 PM, "Robert Metzger" <rm...@apache.org> wrote:
>>
>> Thanks a lot for your efforts!
>>
>> Maybe we can even put a small blog post on the Flink post once its
>> available on docker hub.
>>
>> On Fri, Apr 28, 2017 at 11:00 AM, Ismaël Mejía <ie...@gmail.com> wrote:
>>
>> > Hello,
>> >
>> > I am absolutely happy to see that this is finally happening!
>> > We have a really neat image right now and it is great that it will be
>> > soon so easy to use.
>> >
>> > One extra thing to mention is that Flink will have now two docker
>> > images, one based on debian and the other one based on Alpine as most
>> > official java-based projects do.
>> >
>> > In the future we expect to improve the documentation on how to use the
>> > image with kubernetes and continue improving the actual documentation
>> > with docker. If anyone wants to join to also document something or add
>> > any improvement/feature you need, you are all welcome.
>> >
>> > Finally, I would also like to thank Maximilian Michels which
>> > contributed and reviewed some of my early changes on the image.
>> >
>> > Regards,
>> > Ismaël
>> >
>> > ps. We will 'announce' when the official images are available on
>> > docker hub, so everyone can start to use them.
>> >
>> >
>> >
>> > On Thu, Apr 27, 2017 at 1:38 PM, Patrick Lucas
>> > <pa...@data-artisans.com> wrote:
>> > > I've been informed that images don't make it through the list!
>> > >
>> > > You can see the aforementioned squirrel here
>> > > <https://activerain-store.s3.amazonaws.com/image_store/
>> > uploads/8/7/6/3/9/ar12988558393678.JPG>
>> > > .
>> > >
>> > > --
>> > > Patrick Lucas
>> > >
>> > > On Thu, Apr 27, 2017 at 12:21 PM, Patrick Lucas <
>> > patrick@data-artisans.com>
>> > > wrote:
>> > >
>> > >> As part of an ongoing effort to improve the experience of using Flink
>> > >> on
>> > >> Docker, some work has been done over the last two months to publish
>> > >> official Flink Docker images to Docker Hub. The goal in the short
>> > >> term
>> > is
>> > >> to make running a simple Flink cluster (almost) as easy as running
>> > docker
>> > >> run flink. In the long term, we would like these images to be good
>> > enough
>> > >> to use in production directly, or as base images for use in an
>> > >> existing
>> > >> Docker workflow.
>> > >>
>> > >> Flink 1.2.1 has some fixes over the last few releases that make
>> > >> running
>> > it
>> > >> on Docker nicer—and in some cases, possible. Notably, FLINK-2821
>> > >> <https://issues.apache.org/jira/browse/FLINK-2821> allowed Dockerized
>> > >> Flink to run across multiple hosts, and FLINK-4326
>> > >> <https://issues.apache.org/jira/browse/FLINK-4326> added an option to
>> > run
>> > >> Flink in the foreground, which is greatly preferred when running in
>> > Docker.
>> > >>
>> > >> We (Ismaël Mejía and myself, with some discussion with Stephan Ewen)
>> > >> decided it made sense to bring the actual Dockerfiles outside of the
>> > Apache
>> > >> Flink git repo, primarily to conform with every other Apache project
>> > that
>> > >> has official images, but also because these scripts logically exist
>> > >> decoupled from any particular Flink version. They are still
>> > Apache-licensed
>> > >> and maintained by the community.
>> > >>
>> > >> Please reply here or on the relevant JIRA/GitHub issue if you have
>> > >> questions or feedback.
>> > >>
>> > >> Here's a squirrel in a container:
>> > >>
>> > >>
>> > >>
>> > >> References:
>> > >>
>> > >> - FLINK-3026: Publish the flink docker container to the docker
>> > registry
>> > >> <https://issues.apache.org/jira/browse/FLINK-3026>
>> > >> - Repo for the Dockerfiles and the scripts that generate them
>> > >> <https://github.com/docker-flink/docker-flink>
>> > >> - GitHub PR to add the official images to Docker Hub
>> > >> <https://github.com/docker-library/official-images/pull/2895>
>> > >> - Improvements to the quality of running Flink on Docker to be
>> > >> made
>> > in
>> > >> future Flink releases:
>> > >> - FLINK-6300: PID1 of docker images does not behave correctly
>> > >> <https://issues.apache.org/jira/browse/FLINK-6300>
>> > >> - FLINK-6369: Better support for overlay networks
>> > >> <https://issues.apache.org/jira/browse/FLINK-6369>
>> > >>
>> > >> Thanks to Ismaël Mejía, Jamie Grier, and Stephan Ewen for their
>> > >> contributions.
>> > >>
>> > >> --
>> > >> Patrick Lucas
>> > >>
>> >
Re: Official Flink Docker images
Posted by Ismaël Mejía <ie...@gmail.com>.
Sure we would do it. I will sync with Patrick to do this quickly.
On May 5, 2017 3:45 PM, "Robert Metzger" <rm...@apache.org> wrote:
> Thanks a lot for your efforts!
>
> Maybe we can even put a small blog post on the Flink post once its
> available on docker hub.
>
> On Fri, Apr 28, 2017 at 11:00 AM, Ismaël Mejía <ie...@gmail.com> wrote:
>
> > Hello,
> >
> > I am absolutely happy to see that this is finally happening!
> > We have a really neat image right now and it is great that it will be
> > soon so easy to use.
> >
> > One extra thing to mention is that Flink will have now two docker
> > images, one based on debian and the other one based on Alpine as most
> > official java-based projects do.
> >
> > In the future we expect to improve the documentation on how to use the
> > image with kubernetes and continue improving the actual documentation
> > with docker. If anyone wants to join to also document something or add
> > any improvement/feature you need, you are all welcome.
> >
> > Finally, I would also like to thank Maximilian Michels which
> > contributed and reviewed some of my early changes on the image.
> >
> > Regards,
> > Ismaël
> >
> > ps. We will 'announce' when the official images are available on
> > docker hub, so everyone can start to use them.
> >
> >
> >
> > On Thu, Apr 27, 2017 at 1:38 PM, Patrick Lucas
> > <pa...@data-artisans.com> wrote:
> > > I've been informed that images don't make it through the list!
> > >
> > > You can see the aforementioned squirrel here
> > > <https://activerain-store.s3.amazonaws.com/image_store/
> > uploads/8/7/6/3/9/ar12988558393678.JPG>
> > > .
> > >
> > > --
> > > Patrick Lucas
> > >
> > > On Thu, Apr 27, 2017 at 12:21 PM, Patrick Lucas <
> > patrick@data-artisans.com>
> > > wrote:
> > >
> > >> As part of an ongoing effort to improve the experience of using Flink
> on
> > >> Docker, some work has been done over the last two months to publish
> > >> official Flink Docker images to Docker Hub. The goal in the short term
> > is
> > >> to make running a simple Flink cluster (almost) as easy as running
> > docker
> > >> run flink. In the long term, we would like these images to be good
> > enough
> > >> to use in production directly, or as base images for use in an
> existing
> > >> Docker workflow.
> > >>
> > >> Flink 1.2.1 has some fixes over the last few releases that make
> running
> > it
> > >> on Docker nicer—and in some cases, possible. Notably, FLINK-2821
> > >> <https://issues.apache.org/jira/browse/FLINK-2821> allowed Dockerized
> > >> Flink to run across multiple hosts, and FLINK-4326
> > >> <https://issues.apache.org/jira/browse/FLINK-4326> added an option to
> > run
> > >> Flink in the foreground, which is greatly preferred when running in
> > Docker.
> > >>
> > >> We (Ismaël Mejía and myself, with some discussion with Stephan Ewen)
> > >> decided it made sense to bring the actual Dockerfiles outside of the
> > Apache
> > >> Flink git repo, primarily to conform with every other Apache project
> > that
> > >> has official images, but also because these scripts logically exist
> > >> decoupled from any particular Flink version. They are still
> > Apache-licensed
> > >> and maintained by the community.
> > >>
> > >> Please reply here or on the relevant JIRA/GitHub issue if you have
> > >> questions or feedback.
> > >>
> > >> Here's a squirrel in a container:
> > >>
> > >>
> > >>
> > >> References:
> > >>
> > >> - FLINK-3026: Publish the flink docker container to the docker
> > registry
> > >> <https://issues.apache.org/jira/browse/FLINK-3026>
> > >> - Repo for the Dockerfiles and the scripts that generate them
> > >> <https://github.com/docker-flink/docker-flink>
> > >> - GitHub PR to add the official images to Docker Hub
> > >> <https://github.com/docker-library/official-images/pull/2895>
> > >> - Improvements to the quality of running Flink on Docker to be made
> > in
> > >> future Flink releases:
> > >> - FLINK-6300: PID1 of docker images does not behave correctly
> > >> <https://issues.apache.org/jira/browse/FLINK-6300>
> > >> - FLINK-6369: Better support for overlay networks
> > >> <https://issues.apache.org/jira/browse/FLINK-6369>
> > >>
> > >> Thanks to Ismaël Mejía, Jamie Grier, and Stephan Ewen for their
> > >> contributions.
> > >>
> > >> --
> > >> Patrick Lucas
> > >>
> >
>
Re: Official Flink Docker images
Posted by Robert Metzger <rm...@apache.org>.
Thanks a lot for your efforts!
Maybe we can even put a small blog post on the Flink post once its
available on docker hub.
On Fri, Apr 28, 2017 at 11:00 AM, Ismaël Mejía <ie...@gmail.com> wrote:
> Hello,
>
> I am absolutely happy to see that this is finally happening!
> We have a really neat image right now and it is great that it will be
> soon so easy to use.
>
> One extra thing to mention is that Flink will have now two docker
> images, one based on debian and the other one based on Alpine as most
> official java-based projects do.
>
> In the future we expect to improve the documentation on how to use the
> image with kubernetes and continue improving the actual documentation
> with docker. If anyone wants to join to also document something or add
> any improvement/feature you need, you are all welcome.
>
> Finally, I would also like to thank Maximilian Michels which
> contributed and reviewed some of my early changes on the image.
>
> Regards,
> Ismaël
>
> ps. We will 'announce' when the official images are available on
> docker hub, so everyone can start to use them.
>
>
>
> On Thu, Apr 27, 2017 at 1:38 PM, Patrick Lucas
> <pa...@data-artisans.com> wrote:
> > I've been informed that images don't make it through the list!
> >
> > You can see the aforementioned squirrel here
> > <https://activerain-store.s3.amazonaws.com/image_store/
> uploads/8/7/6/3/9/ar12988558393678.JPG>
> > .
> >
> > --
> > Patrick Lucas
> >
> > On Thu, Apr 27, 2017 at 12:21 PM, Patrick Lucas <
> patrick@data-artisans.com>
> > wrote:
> >
> >> As part of an ongoing effort to improve the experience of using Flink on
> >> Docker, some work has been done over the last two months to publish
> >> official Flink Docker images to Docker Hub. The goal in the short term
> is
> >> to make running a simple Flink cluster (almost) as easy as running
> docker
> >> run flink. In the long term, we would like these images to be good
> enough
> >> to use in production directly, or as base images for use in an existing
> >> Docker workflow.
> >>
> >> Flink 1.2.1 has some fixes over the last few releases that make running
> it
> >> on Docker nicer—and in some cases, possible. Notably, FLINK-2821
> >> <https://issues.apache.org/jira/browse/FLINK-2821> allowed Dockerized
> >> Flink to run across multiple hosts, and FLINK-4326
> >> <https://issues.apache.org/jira/browse/FLINK-4326> added an option to
> run
> >> Flink in the foreground, which is greatly preferred when running in
> Docker.
> >>
> >> We (Ismaël Mejía and myself, with some discussion with Stephan Ewen)
> >> decided it made sense to bring the actual Dockerfiles outside of the
> Apache
> >> Flink git repo, primarily to conform with every other Apache project
> that
> >> has official images, but also because these scripts logically exist
> >> decoupled from any particular Flink version. They are still
> Apache-licensed
> >> and maintained by the community.
> >>
> >> Please reply here or on the relevant JIRA/GitHub issue if you have
> >> questions or feedback.
> >>
> >> Here's a squirrel in a container:
> >>
> >>
> >>
> >> References:
> >>
> >> - FLINK-3026: Publish the flink docker container to the docker
> registry
> >> <https://issues.apache.org/jira/browse/FLINK-3026>
> >> - Repo for the Dockerfiles and the scripts that generate them
> >> <https://github.com/docker-flink/docker-flink>
> >> - GitHub PR to add the official images to Docker Hub
> >> <https://github.com/docker-library/official-images/pull/2895>
> >> - Improvements to the quality of running Flink on Docker to be made
> in
> >> future Flink releases:
> >> - FLINK-6300: PID1 of docker images does not behave correctly
> >> <https://issues.apache.org/jira/browse/FLINK-6300>
> >> - FLINK-6369: Better support for overlay networks
> >> <https://issues.apache.org/jira/browse/FLINK-6369>
> >>
> >> Thanks to Ismaël Mejía, Jamie Grier, and Stephan Ewen for their
> >> contributions.
> >>
> >> --
> >> Patrick Lucas
> >>
>
Re: Official Flink Docker images
Posted by Ismaël Mejía <ie...@gmail.com>.
Hello,
I am absolutely happy to see that this is finally happening!
We have a really neat image right now and it is great that it will be
soon so easy to use.
One extra thing to mention is that Flink will have now two docker
images, one based on debian and the other one based on Alpine as most
official java-based projects do.
In the future we expect to improve the documentation on how to use the
image with kubernetes and continue improving the actual documentation
with docker. If anyone wants to join to also document something or add
any improvement/feature you need, you are all welcome.
Finally, I would also like to thank Maximilian Michels which
contributed and reviewed some of my early changes on the image.
Regards,
Ismaël
ps. We will 'announce' when the official images are available on
docker hub, so everyone can start to use them.
On Thu, Apr 27, 2017 at 1:38 PM, Patrick Lucas
<pa...@data-artisans.com> wrote:
> I've been informed that images don't make it through the list!
>
> You can see the aforementioned squirrel here
> <https://activerain-store.s3.amazonaws.com/image_store/uploads/8/7/6/3/9/ar12988558393678.JPG>
> .
>
> --
> Patrick Lucas
>
> On Thu, Apr 27, 2017 at 12:21 PM, Patrick Lucas <pa...@data-artisans.com>
> wrote:
>
>> As part of an ongoing effort to improve the experience of using Flink on
>> Docker, some work has been done over the last two months to publish
>> official Flink Docker images to Docker Hub. The goal in the short term is
>> to make running a simple Flink cluster (almost) as easy as running docker
>> run flink. In the long term, we would like these images to be good enough
>> to use in production directly, or as base images for use in an existing
>> Docker workflow.
>>
>> Flink 1.2.1 has some fixes over the last few releases that make running it
>> on Docker nicer—and in some cases, possible. Notably, FLINK-2821
>> <https://issues.apache.org/jira/browse/FLINK-2821> allowed Dockerized
>> Flink to run across multiple hosts, and FLINK-4326
>> <https://issues.apache.org/jira/browse/FLINK-4326> added an option to run
>> Flink in the foreground, which is greatly preferred when running in Docker.
>>
>> We (Ismaël Mejía and myself, with some discussion with Stephan Ewen)
>> decided it made sense to bring the actual Dockerfiles outside of the Apache
>> Flink git repo, primarily to conform with every other Apache project that
>> has official images, but also because these scripts logically exist
>> decoupled from any particular Flink version. They are still Apache-licensed
>> and maintained by the community.
>>
>> Please reply here or on the relevant JIRA/GitHub issue if you have
>> questions or feedback.
>>
>> Here's a squirrel in a container:
>>
>>
>>
>> References:
>>
>> - FLINK-3026: Publish the flink docker container to the docker registry
>> <https://issues.apache.org/jira/browse/FLINK-3026>
>> - Repo for the Dockerfiles and the scripts that generate them
>> <https://github.com/docker-flink/docker-flink>
>> - GitHub PR to add the official images to Docker Hub
>> <https://github.com/docker-library/official-images/pull/2895>
>> - Improvements to the quality of running Flink on Docker to be made in
>> future Flink releases:
>> - FLINK-6300: PID1 of docker images does not behave correctly
>> <https://issues.apache.org/jira/browse/FLINK-6300>
>> - FLINK-6369: Better support for overlay networks
>> <https://issues.apache.org/jira/browse/FLINK-6369>
>>
>> Thanks to Ismaël Mejía, Jamie Grier, and Stephan Ewen for their
>> contributions.
>>
>> --
>> Patrick Lucas
>>
Re: Official Flink Docker images
Posted by Patrick Lucas <pa...@data-artisans.com>.
I've been informed that images don't make it through the list!
You can see the aforementioned squirrel here
<https://activerain-store.s3.amazonaws.com/image_store/uploads/8/7/6/3/9/ar12988558393678.JPG>
.
--
Patrick Lucas
On Thu, Apr 27, 2017 at 12:21 PM, Patrick Lucas <pa...@data-artisans.com>
wrote:
> As part of an ongoing effort to improve the experience of using Flink on
> Docker, some work has been done over the last two months to publish
> official Flink Docker images to Docker Hub. The goal in the short term is
> to make running a simple Flink cluster (almost) as easy as running docker
> run flink. In the long term, we would like these images to be good enough
> to use in production directly, or as base images for use in an existing
> Docker workflow.
>
> Flink 1.2.1 has some fixes over the last few releases that make running it
> on Docker nicer—and in some cases, possible. Notably, FLINK-2821
> <https://issues.apache.org/jira/browse/FLINK-2821> allowed Dockerized
> Flink to run across multiple hosts, and FLINK-4326
> <https://issues.apache.org/jira/browse/FLINK-4326> added an option to run
> Flink in the foreground, which is greatly preferred when running in Docker.
>
> We (Ismaël Mejía and myself, with some discussion with Stephan Ewen)
> decided it made sense to bring the actual Dockerfiles outside of the Apache
> Flink git repo, primarily to conform with every other Apache project that
> has official images, but also because these scripts logically exist
> decoupled from any particular Flink version. They are still Apache-licensed
> and maintained by the community.
>
> Please reply here or on the relevant JIRA/GitHub issue if you have
> questions or feedback.
>
> Here's a squirrel in a container:
>
>
>
> References:
>
> - FLINK-3026: Publish the flink docker container to the docker registry
> <https://issues.apache.org/jira/browse/FLINK-3026>
> - Repo for the Dockerfiles and the scripts that generate them
> <https://github.com/docker-flink/docker-flink>
> - GitHub PR to add the official images to Docker Hub
> <https://github.com/docker-library/official-images/pull/2895>
> - Improvements to the quality of running Flink on Docker to be made in
> future Flink releases:
> - FLINK-6300: PID1 of docker images does not behave correctly
> <https://issues.apache.org/jira/browse/FLINK-6300>
> - FLINK-6369: Better support for overlay networks
> <https://issues.apache.org/jira/browse/FLINK-6369>
>
> Thanks to Ismaël Mejía, Jamie Grier, and Stephan Ewen for their
> contributions.
>
> --
> Patrick Lucas
>