You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by rm...@apache.org on 2020/08/20 11:44:22 UTC
[flink-web] 01/02: [blog] Add post about flink on docker
This is an automated email from the ASF dual-hosted git repository.
rmetzger pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git
commit e7376f0885cfd598679f8c1b26faa5d2fa435900
Author: Robert Metzger <rm...@apache.org>
AuthorDate: Tue Aug 18 21:22:41 2020 +0200
[blog] Add post about flink on docker
This closes #370
---
_posts/2020-08-20-flink-docker.md | 98 +++++++++++++++++++++++++++++++++
img/blog/flink-docker/flink-docker.gif | Bin 0 -> 4844328 bytes
2 files changed, 98 insertions(+)
diff --git a/_posts/2020-08-20-flink-docker.md b/_posts/2020-08-20-flink-docker.md
new file mode 100644
index 0000000..911f29e
--- /dev/null
+++ b/_posts/2020-08-20-flink-docker.md
@@ -0,0 +1,98 @@
+---
+layout: post
+title: "The State of Flink on Docker"
+date: 2020-08-20T00:00:00.000Z
+authors:
+- rmetzger:
+ name: "Robert Metzger"
+ twitter: rmetzger_
+categories: news
+
+excerpt: This blog post gives an update on the recent developments of Flink's support for Docker.
+---
+
+With over 50 million downloads from Docker Hub, the Flink docker images are a very popular deployment option.
+
+The Flink community recently put some effort into improving the Docker experience for our users with the goal to reduce confusion and improve usability.
+
+
+Let's quickly break down the recent improvements:
+
+- Reduce confusion: Flink used to have 2 Dockerfiles and a 3rd file maintained outside of the official repository — all with different features and varying stability. Now, we have one central place for all images: [apache/flink-docker](https://github.com/apache/flink-docker).
+
+ Here, we keep all the Dockerfiles for the different releases. Check out the [detailed readme](https://github.com/apache/flink-docker/blob/master/README.md) of that repository for further explanation on the different branches, as well as the [Flink Improvement Proposal (FLIP-111)](https://cwiki.apache.org/confluence/display/FLINK/FLIP-111%3A+Docker+image+unification) that contains the detailed planning.
+
+ The `apache/flink-docker` repository also seeds the [official Flink image on Docker Hub](https://hub.docker.com/_/flink).
+
+- Improve Usability: The Dockerfiles are used for various purposes: [Native Docker deployments](https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/docker.html), [Flink on Kubernetes](https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/native_kubernetes.html), the (unofficial) [Flink helm example](https://github.com/docker-flink/examples) and the project's [internal end to end tests](https://github.com/apache/flink/tree/master/flink-end-to-end-tests). [...]
+
+ The new images support [passing configuration variables](https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/docker.html#configure-options) via a `FLINK_PROPERTIES` environment variable. Users can [enable default plugins](https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/docker.html#using-plugins) with the `ENABLE_BUILT_IN_PLUGINS` environment variable. The images also allow loading custom jar paths and configuration files.
+
+Looking into the future, there are already some interesting potential improvements lined up:
+
+- [Java 11 Docker images](https://issues.apache.org/jira/browse/FLINK-16260) (already completed)
+- [Use vanilla docker-entrypoint with flink-kubernetes](https://issues.apache.org/jira/browse/FLINK-15793) (in progress)
+- [History server support](https://issues.apache.org/jira/browse/FLINK-17167)
+- [Support for OpenShift](https://issues.apache.org/jira/browse/FLINK-15587)
+
+## How do I get started?
+
+This is a short tutorial on [how to start a Flink Session Cluster](https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/docker.html#start-a-session-cluster) with Docker.
+
+A *Flink Session cluster* can be used to run multiple jobs. Each job needs to be submitted to the cluster after it has been deployed. To deploy a *Flink Session cluster* with Docker, you need to start a *JobManager* container. To enable communication between the containers, we first set a required Flink configuration property and create a network:
+
+```
+FLINK_PROPERTIES="jobmanager.rpc.address: jobmanager"
+docker network create flink-network
+```
+
+Then we launch the JobManager:
+
+```
+docker run \
+ --rm \
+ --name=jobmanager \
+ --network flink-network \
+ -p 8081:8081 \
+ --env FLINK_PROPERTIES="${FLINK_PROPERTIES}" \
+ flink:1.11.1 jobmanager
+```
+and one or more *TaskManager* containers:
+
+```
+docker run \
+ --rm \
+ --name=taskmanager \
+ --network flink-network \
+ --env FLINK_PROPERTIES="${FLINK_PROPERTIES}" \
+ flink:1.11.1 taskmanager
+```
+
+You now have a fully functional Flink cluster running! You can access the the web front end here: [localhost:8081](http://localhost:8081/).
+
+Let's now submit one of Flink's example jobs:
+
+```bash
+# 1: (optional) Download the Flink distribution, and unpack it
+wget https://archive.apache.org/dist/flink/flink-1.11.1/flink-1.11.1-bin-scala_2.12.tgz
+tar xf flink-1.11.1-bin-scala_2.12.tgz
+cd flink-1.11.1
+
+# 2: Start the Flink job
+./bin/flink run ./examples/streaming/TopSpeedWindowing.jar
+```
+
+The main steps of the tutorial are also recorded in this short screencast:
+
+<center>
+<img src="{{ site.baseurl }}/img/blog/flink-docker/flink-docker.gif" width="882px" height="730px" alt="Demo video"/>
+</center>
+
+
+**Next steps**: Now that you've successfully completed this tutorial, we recommend you checking out the full [Flink on Docker documentation](https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/docker.html) for implementing more advanced deployment scenarios, such as Job Clusters, Docker Compose or our native Kubernetes integration.
+
+
+## Conclusion
+
+We encourage all readers to try out Flink on Docker to provide the community with feedback to further improve the experience.
+Please refer to the user@flink.apache.org ([remember to subscribe first](https://flink.apache.org/community.html#how-to-subscribe-to-a-mailing-list)) for general questions and our [issue tracker](https://issues.apache.org/jira/issues/?jql=project+%3D+FLINK+AND+component+%3D+flink-docker) for specific bugs or improvements, or [ideas for contributions](https://flink.apache.org/contributing/how-to-contribute.html)!
diff --git a/img/blog/flink-docker/flink-docker.gif b/img/blog/flink-docker/flink-docker.gif
new file mode 100644
index 0000000..3495acf
Binary files /dev/null and b/img/blog/flink-docker/flink-docker.gif differ