You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by sj...@apache.org on 2020/07/15 14:48:44 UTC
[flink-web] branch asf-site updated (0ae6413 -> 33bbdaa)
This is an automated email from the ASF dual-hosted git repository.
sjwiesman pushed a change to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git.
from 0ae6413 rebuild website
add e2510cd Application Mode blogpost
add d0d8232 Making the blogpost compilable and adding a few adjustments.
new 33bbdaa rebuild website
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
_posts/2020-07-14-application-mode.md | 282 +++++++++++
content/blog/page10/index.html | 40 +-
content/blog/page11/index.html | 40 +-
content/blog/page12/index.html | 25 +
content/blog/page2/index.html | 36 +-
content/blog/page3/index.html | 36 +-
content/blog/page4/index.html | 42 +-
content/blog/page5/index.html | 46 +-
content/blog/page6/index.html | 42 +-
content/blog/page7/index.html | 40 +-
content/blog/page8/index.html | 40 +-
content/blog/page9/index.html | 40 +-
.../2020-07-14-application-mode/application.png | Bin 0 -> 94513 bytes
.../session-per-job.png | Bin 0 -> 184784 bytes
content/news/2020/07/14/application-mode.html | 540 +++++++++++++++++++++
.../2020-07-14-application-mode/application.png | Bin 0 -> 94513 bytes
.../session-per-job.png | Bin 0 -> 184784 bytes
17 files changed, 1097 insertions(+), 152 deletions(-)
create mode 100644 _posts/2020-07-14-application-mode.md
create mode 100644 content/img/blog/2020-07-14-application-mode/application.png
create mode 100644 content/img/blog/2020-07-14-application-mode/session-per-job.png
create mode 100644 content/news/2020/07/14/application-mode.html
create mode 100644 img/blog/2020-07-14-application-mode/application.png
create mode 100644 img/blog/2020-07-14-application-mode/session-per-job.png
[flink-web] 01/01: rebuild website
Posted by sj...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
sjwiesman pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git
commit 33bbdaa2a16e1adeef4e8115a8eb33b664b74c5c
Author: Seth Wiesman <sj...@gmail.com>
AuthorDate: Wed Jul 15 09:46:29 2020 -0500
rebuild website
---
_posts/2020-07-14-application-mode.md | 3 +-
content/blog/page10/index.html | 40 +-
content/blog/page11/index.html | 40 +-
content/blog/page12/index.html | 25 +
content/blog/page2/index.html | 36 +-
content/blog/page3/index.html | 36 +-
content/blog/page4/index.html | 42 +-
content/blog/page5/index.html | 46 +-
content/blog/page6/index.html | 42 +-
content/blog/page7/index.html | 40 +-
content/blog/page8/index.html | 40 +-
content/blog/page9/index.html | 40 +-
.../2020-07-14-application-mode/application.png | Bin 0 -> 94513 bytes
.../session-per-job.png | Bin 0 -> 184784 bytes
content/news/2020/07/14/application-mode.html | 540 +++++++++++++++++++++
15 files changed, 817 insertions(+), 153 deletions(-)
diff --git a/_posts/2020-07-14-application-mode.md b/_posts/2020-07-14-application-mode.md
index c002f89..54093ee 100644
--- a/_posts/2020-07-14-application-mode.md
+++ b/_posts/2020-07-14-application-mode.md
@@ -1,7 +1,8 @@
---
layout: post
-title: "Lightweight application submission with the Application Mode"
+title: "Application Deployment in Flink: Current State and the new Application Mode"
date: 2020-07-14T08:00:00.000Z
+category: news
authors:
- kostas:
name: "Kostas Kloudas"
diff --git a/content/blog/page10/index.html b/content/blog/page10/index.html
index 9da2914..b590cd0 100644
--- a/content/blog/page10/index.html
+++ b/content/blog/page10/index.html
@@ -196,6 +196,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2016/04/06/release-1.0.1.html">Flink 1.0.1 Released</a></h2>
+
+ <p>06 Apr 2016
+ </p>
+
+ <p><p>Today, the Flink community released Flink version <strong>1.0.1</strong>, the first bugfix release of the 1.0 series.</p>
+
+</p>
+
+ <p><a href="/news/2016/04/06/release-1.0.1.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2016/03/08/release-1.0.0.html">Announcing Apache Flink 1.0.0</a></h2>
<p>08 Mar 2016
@@ -328,21 +343,6 @@ Apache Flink started.</p>
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2015/09/01/release-0.9.1.html">Apache Flink 0.9.1 available</a></h2>
-
- <p>01 Sep 2015
- </p>
-
- <p><p>The Flink community is happy to announce that Flink 0.9.1 is now available.</p>
-
-</p>
-
- <p><a href="/news/2015/09/01/release-0.9.1.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -375,6 +375,16 @@ Apache Flink started.</p>
<ul id="markdown-toc">
+ <li><a href="/news/2020/07/14/application-mode.html">Application Deployment in Flink: current state and the new Application Mode</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/07/06/release-1.11.0.html">Apache Flink 1.11.0 Release Announcement</a></li>
diff --git a/content/blog/page11/index.html b/content/blog/page11/index.html
index ce6d1cc..69b329b 100644
--- a/content/blog/page11/index.html
+++ b/content/blog/page11/index.html
@@ -196,6 +196,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2015/09/01/release-0.9.1.html">Apache Flink 0.9.1 available</a></h2>
+
+ <p>01 Sep 2015
+ </p>
+
+ <p><p>The Flink community is happy to announce that Flink 0.9.1 is now available.</p>
+
+</p>
+
+ <p><a href="/news/2015/09/01/release-0.9.1.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2015/08/24/introducing-flink-gelly.html">Introducing Gelly: Graph Processing with Apache Flink</a></h2>
<p>24 Aug 2015
@@ -340,21 +355,6 @@ and offers a new API including definition of flexible windows.</p>
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2015/02/04/january-in-flink.html">January 2015 in the Flink community</a></h2>
-
- <p>04 Feb 2015
- </p>
-
- <p><p>Happy 2015! Here is a (hopefully digestible) summary of what happened last month in the Flink community.</p>
-
-</p>
-
- <p><a href="/news/2015/02/04/january-in-flink.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -387,6 +387,16 @@ and offers a new API including definition of flexible windows.</p>
<ul id="markdown-toc">
+ <li><a href="/news/2020/07/14/application-mode.html">Application Deployment in Flink: current state and the new Application Mode</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/07/06/release-1.11.0.html">Apache Flink 1.11.0 Release Announcement</a></li>
diff --git a/content/blog/page12/index.html b/content/blog/page12/index.html
index 27f2425..2ef3694 100644
--- a/content/blog/page12/index.html
+++ b/content/blog/page12/index.html
@@ -196,6 +196,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2015/02/04/january-in-flink.html">January 2015 in the Flink community</a></h2>
+
+ <p>04 Feb 2015
+ </p>
+
+ <p><p>Happy 2015! Here is a (hopefully digestible) summary of what happened last month in the Flink community.</p>
+
+</p>
+
+ <p><a href="/news/2015/02/04/january-in-flink.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2015/01/21/release-0.8.html">Apache Flink 0.8.0 available</a></h2>
<p>21 Jan 2015
@@ -335,6 +350,16 @@ academic and open source project that Flink originates from.</p>
<ul id="markdown-toc">
+ <li><a href="/news/2020/07/14/application-mode.html">Application Deployment in Flink: current state and the new Application Mode</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/07/06/release-1.11.0.html">Apache Flink 1.11.0 Release Announcement</a></li>
diff --git a/content/blog/page2/index.html b/content/blog/page2/index.html
index f634033..f9b0845 100644
--- a/content/blog/page2/index.html
+++ b/content/blog/page2/index.html
@@ -196,6 +196,19 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2020/04/21/memory-management-improvements-flink-1.10.html">Memory Management Improvements with Apache Flink 1.10</a></h2>
+
+ <p>21 Apr 2020
+ Andrey Zagrebin </p>
+
+ <p>This post discusses the recent changes to the memory model of the Task Managers and configuration options for your Flink applications in Flink 1.10.</p>
+
+ <p><a href="/news/2020/04/21/memory-management-improvements-flink-1.10.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2020/04/15/flink-serialization-tuning-vol-1.html">Flink Serialization Tuning Vol. 1: Choosing your Serializer — if you can</a></h2>
<p>15 Apr 2020
@@ -319,19 +332,6 @@ This release marks a big milestone: Stateful Functions 2.0 is not only an API up
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2020/02/07/a-guide-for-unit-testing-in-apache-flink.html">A Guide for Unit Testing in Apache Flink</a></h2>
-
- <p>07 Feb 2020
- Kartik Khare (<a href="https://twitter.com/khare_khote">@khare_khote</a>)</p>
-
- <p>This post provides a detailed guide for unit testing of Apache Flink applications.</p>
-
- <p><a href="/news/2020/02/07/a-guide-for-unit-testing-in-apache-flink.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -364,6 +364,16 @@ This release marks a big milestone: Stateful Functions 2.0 is not only an API up
<ul id="markdown-toc">
+ <li><a href="/news/2020/07/14/application-mode.html">Application Deployment in Flink: current state and the new Application Mode</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/07/06/release-1.11.0.html">Apache Flink 1.11.0 Release Announcement</a></li>
diff --git a/content/blog/page3/index.html b/content/blog/page3/index.html
index 1a1760b..2292e7e 100644
--- a/content/blog/page3/index.html
+++ b/content/blog/page3/index.html
@@ -196,6 +196,19 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2020/02/07/a-guide-for-unit-testing-in-apache-flink.html">A Guide for Unit Testing in Apache Flink</a></h2>
+
+ <p>07 Feb 2020
+ Kartik Khare (<a href="https://twitter.com/khare_khote">@khare_khote</a>)</p>
+
+ <p>This post provides a detailed guide for unit testing of Apache Flink applications.</p>
+
+ <p><a href="/news/2020/02/07/a-guide-for-unit-testing-in-apache-flink.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2020/01/30/release-1.9.2.html">Apache Flink 1.9.2 Released</a></h2>
<p>30 Jan 2020
@@ -320,19 +333,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2019/09/10/community-update.html">Flink Community Update - September'19</a></h2>
-
- <p>10 Sep 2019
- Marta Paes (<a href="https://twitter.com/morsapaes">@morsapaes</a>)</p>
-
- <p>This has been an exciting, fast-paced year for the Apache Flink community. But with over 10k messages across the mailing lists, 3k Jira tickets and 2k pull requests, it is not easy to keep up with the latest state of the project. Plus everything happening around it. With that in mind, we want to bring back regular community updates to the Flink blog.</p>
-
- <p><a href="/news/2019/09/10/community-update.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -365,6 +365,16 @@
<ul id="markdown-toc">
+ <li><a href="/news/2020/07/14/application-mode.html">Application Deployment in Flink: current state and the new Application Mode</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/07/06/release-1.11.0.html">Apache Flink 1.11.0 Release Announcement</a></li>
diff --git a/content/blog/page4/index.html b/content/blog/page4/index.html
index f3b4b2f..23e1499 100644
--- a/content/blog/page4/index.html
+++ b/content/blog/page4/index.html
@@ -196,6 +196,19 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2019/09/10/community-update.html">Flink Community Update - September'19</a></h2>
+
+ <p>10 Sep 2019
+ Marta Paes (<a href="https://twitter.com/morsapaes">@morsapaes</a>)</p>
+
+ <p>This has been an exciting, fast-paced year for the Apache Flink community. But with over 10k messages across the mailing lists, 3k Jira tickets and 2k pull requests, it is not easy to keep up with the latest state of the project. Plus everything happening around it. With that in mind, we want to bring back regular community updates to the Flink blog.</p>
+
+ <p><a href="/news/2019/09/10/community-update.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2019/08/22/release-1.9.0.html">Apache Flink 1.9.0 Release Announcement</a></h2>
<p>22 Aug 2019
@@ -319,25 +332,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2019/04/09/release-1.8.0.html">Apache Flink 1.8.0 Release Announcement</a></h2>
-
- <p>09 Apr 2019
- Aljoscha Krettek (<a href="https://twitter.com/aljoscha">@aljoscha</a>)</p>
-
- <p><p>The Apache Flink community is pleased to announce Apache Flink 1.8.0. The
-latest release includes more than 420 resolved issues and some exciting
-additions to Flink that we describe in the following sections of this post.
-Please check the <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12344274">complete changelog</a>
-for more details.</p>
-
-</p>
-
- <p><a href="/news/2019/04/09/release-1.8.0.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -370,6 +364,16 @@ for more details.</p>
<ul id="markdown-toc">
+ <li><a href="/news/2020/07/14/application-mode.html">Application Deployment in Flink: current state and the new Application Mode</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/07/06/release-1.11.0.html">Apache Flink 1.11.0 Release Announcement</a></li>
diff --git a/content/blog/page5/index.html b/content/blog/page5/index.html
index 85fc481..5d4eda4 100644
--- a/content/blog/page5/index.html
+++ b/content/blog/page5/index.html
@@ -196,6 +196,25 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2019/04/09/release-1.8.0.html">Apache Flink 1.8.0 Release Announcement</a></h2>
+
+ <p>09 Apr 2019
+ Aljoscha Krettek (<a href="https://twitter.com/aljoscha">@aljoscha</a>)</p>
+
+ <p><p>The Apache Flink community is pleased to announce Apache Flink 1.8.0. The
+latest release includes more than 420 resolved issues and some exciting
+additions to Flink that we describe in the following sections of this post.
+Please check the <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12344274">complete changelog</a>
+for more details.</p>
+
+</p>
+
+ <p><a href="/news/2019/04/09/release-1.8.0.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/features/2019/03/11/prometheus-monitoring.html">Flink and Prometheus: Cloud-native monitoring of streaming applications</a></h2>
<p>11 Mar 2019
@@ -322,23 +341,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2018/11/30/release-1.7.0.html">Apache Flink 1.7.0 Release Announcement</a></h2>
-
- <p>30 Nov 2018
- Till Rohrmann (<a href="https://twitter.com/stsffap">@stsffap</a>)</p>
-
- <p><p>The Apache Flink community is pleased to announce Apache Flink 1.7.0.
-The latest release includes more than 420 resolved issues and some exciting additions to Flink that we describe in the following sections of this post.
-Please check the <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12343585">complete changelog</a> for more details.</p>
-
-</p>
-
- <p><a href="/news/2018/11/30/release-1.7.0.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -371,6 +373,16 @@ Please check the <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa
<ul id="markdown-toc">
+ <li><a href="/news/2020/07/14/application-mode.html">Application Deployment in Flink: current state and the new Application Mode</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/07/06/release-1.11.0.html">Apache Flink 1.11.0 Release Announcement</a></li>
diff --git a/content/blog/page6/index.html b/content/blog/page6/index.html
index 8d3489c..e001b22 100644
--- a/content/blog/page6/index.html
+++ b/content/blog/page6/index.html
@@ -196,6 +196,23 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2018/11/30/release-1.7.0.html">Apache Flink 1.7.0 Release Announcement</a></h2>
+
+ <p>30 Nov 2018
+ Till Rohrmann (<a href="https://twitter.com/stsffap">@stsffap</a>)</p>
+
+ <p><p>The Apache Flink community is pleased to announce Apache Flink 1.7.0.
+The latest release includes more than 420 resolved issues and some exciting additions to Flink that we describe in the following sections of this post.
+Please check the <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12343585">complete changelog</a> for more details.</p>
+
+</p>
+
+ <p><a href="/news/2018/11/30/release-1.7.0.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2018/10/29/release-1.6.2.html">Apache Flink 1.6.2 Released</a></h2>
<p>29 Oct 2018
@@ -330,21 +347,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2018/03/15/release-1.3.3.html">Apache Flink 1.3.3 Released</a></h2>
-
- <p>15 Mar 2018
- </p>
-
- <p><p>The Apache Flink community released the third bugfix version of the Apache Flink 1.3 series.</p>
-
-</p>
-
- <p><a href="/news/2018/03/15/release-1.3.3.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -377,6 +379,16 @@
<ul id="markdown-toc">
+ <li><a href="/news/2020/07/14/application-mode.html">Application Deployment in Flink: current state and the new Application Mode</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/07/06/release-1.11.0.html">Apache Flink 1.11.0 Release Announcement</a></li>
diff --git a/content/blog/page7/index.html b/content/blog/page7/index.html
index 0c80454..a1eba91 100644
--- a/content/blog/page7/index.html
+++ b/content/blog/page7/index.html
@@ -196,6 +196,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2018/03/15/release-1.3.3.html">Apache Flink 1.3.3 Released</a></h2>
+
+ <p>15 Mar 2018
+ </p>
+
+ <p><p>The Apache Flink community released the third bugfix version of the Apache Flink 1.3 series.</p>
+
+</p>
+
+ <p><a href="/news/2018/03/15/release-1.3.3.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2018/03/08/release-1.4.2.html">Apache Flink 1.4.2 Released</a></h2>
<p>08 Mar 2018
@@ -327,21 +342,6 @@ what’s coming in Flink 1.4.0 as well as a preview of what the Flink community
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2017/06/23/release-1.3.1.html">Apache Flink 1.3.1 Released</a></h2>
-
- <p>23 Jun 2017
- </p>
-
- <p><p>The Apache Flink community released the first bugfix version of the Apache Flink 1.3 series.</p>
-
-</p>
-
- <p><a href="/news/2017/06/23/release-1.3.1.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -374,6 +374,16 @@ what’s coming in Flink 1.4.0 as well as a preview of what the Flink community
<ul id="markdown-toc">
+ <li><a href="/news/2020/07/14/application-mode.html">Application Deployment in Flink: current state and the new Application Mode</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/07/06/release-1.11.0.html">Apache Flink 1.11.0 Release Announcement</a></li>
diff --git a/content/blog/page8/index.html b/content/blog/page8/index.html
index c5e623e..bbe19d0 100644
--- a/content/blog/page8/index.html
+++ b/content/blog/page8/index.html
@@ -196,6 +196,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2017/06/23/release-1.3.1.html">Apache Flink 1.3.1 Released</a></h2>
+
+ <p>23 Jun 2017
+ </p>
+
+ <p><p>The Apache Flink community released the first bugfix version of the Apache Flink 1.3 series.</p>
+
+</p>
+
+ <p><a href="/news/2017/06/23/release-1.3.1.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2017/06/01/release-1.3.0.html">Apache Flink 1.3.0 Release Announcement</a></h2>
<p>01 Jun 2017 by Robert Metzger (<a href="https://twitter.com/">@rmetzger_</a>)
@@ -323,21 +338,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2016/10/12/release-1.1.3.html">Apache Flink 1.1.3 Released</a></h2>
-
- <p>12 Oct 2016
- </p>
-
- <p><p>The Apache Flink community released the next bugfix version of the Apache Flink 1.1. series.</p>
-
-</p>
-
- <p><a href="/news/2016/10/12/release-1.1.3.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -370,6 +370,16 @@
<ul id="markdown-toc">
+ <li><a href="/news/2020/07/14/application-mode.html">Application Deployment in Flink: current state and the new Application Mode</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/07/06/release-1.11.0.html">Apache Flink 1.11.0 Release Announcement</a></li>
diff --git a/content/blog/page9/index.html b/content/blog/page9/index.html
index 406ccf6..268693e 100644
--- a/content/blog/page9/index.html
+++ b/content/blog/page9/index.html
@@ -196,6 +196,21 @@
<!-- Blog posts -->
<article>
+ <h2 class="blog-title"><a href="/news/2016/10/12/release-1.1.3.html">Apache Flink 1.1.3 Released</a></h2>
+
+ <p>12 Oct 2016
+ </p>
+
+ <p><p>The Apache Flink community released the next bugfix version of the Apache Flink 1.1. series.</p>
+
+</p>
+
+ <p><a href="/news/2016/10/12/release-1.1.3.html">Continue reading »</a></p>
+ </article>
+
+ <hr>
+
+ <article>
<h2 class="blog-title"><a href="/news/2016/09/05/release-1.1.2.html">Apache Flink 1.1.2 Released</a></h2>
<p>05 Sep 2016
@@ -327,21 +342,6 @@
<hr>
- <article>
- <h2 class="blog-title"><a href="/news/2016/04/06/release-1.0.1.html">Flink 1.0.1 Released</a></h2>
-
- <p>06 Apr 2016
- </p>
-
- <p><p>Today, the Flink community released Flink version <strong>1.0.1</strong>, the first bugfix release of the 1.0 series.</p>
-
-</p>
-
- <p><a href="/news/2016/04/06/release-1.0.1.html">Continue reading »</a></p>
- </article>
-
- <hr>
-
<!-- Pagination links -->
@@ -374,6 +374,16 @@
<ul id="markdown-toc">
+ <li><a href="/news/2020/07/14/application-mode.html">Application Deployment in Flink: current state and the new Application Mode</a></li>
+
+
+
+
+
+
+
+
+
<li><a href="/news/2020/07/06/release-1.11.0.html">Apache Flink 1.11.0 Release Announcement</a></li>
diff --git a/content/img/blog/2020-07-14-application-mode/application.png b/content/img/blog/2020-07-14-application-mode/application.png
new file mode 100644
index 0000000..ce13938
Binary files /dev/null and b/content/img/blog/2020-07-14-application-mode/application.png differ
diff --git a/content/img/blog/2020-07-14-application-mode/session-per-job.png b/content/img/blog/2020-07-14-application-mode/session-per-job.png
new file mode 100644
index 0000000..dea7ca7
Binary files /dev/null and b/content/img/blog/2020-07-14-application-mode/session-per-job.png differ
diff --git a/content/news/2020/07/14/application-mode.html b/content/news/2020/07/14/application-mode.html
new file mode 100644
index 0000000..366d0e8
--- /dev/null
+++ b/content/news/2020/07/14/application-mode.html
@@ -0,0 +1,540 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>Apache Flink: Application Deployment in Flink: current state and the new Application Mode</title>
+ <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+ <link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+ <!-- Bootstrap -->
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.4.1/css/bootstrap.min.css">
+ <link rel="stylesheet" href="/css/flink.css">
+ <link rel="stylesheet" href="/css/syntax.css">
+
+ <!-- Blog RSS feed -->
+ <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" />
+
+ <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+ <!-- We need to load Jquery in the header for custom google analytics event tracking-->
+ <script src="/js/jquery.min.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+ <body>
+
+
+ <!-- Main content. -->
+ <div class="container">
+ <div class="row">
+
+
+ <div id="sidebar" class="col-sm-3">
+
+
+<!-- Top navbar. -->
+ <nav class="navbar navbar-default">
+ <!-- The logo. -->
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <div class="navbar-logo">
+ <a href="/">
+ <img alt="Apache Flink" src="/img/flink-header-logo.svg" width="147px" height="73px">
+ </a>
+ </div>
+ </div><!-- /.navbar-header -->
+
+ <!-- The navigation links. -->
+ <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
+ <ul class="nav navbar-nav navbar-main">
+
+ <!-- First menu section explains visitors what Flink is -->
+
+ <!-- What is Stream Processing? -->
+ <!--
+ <li><a href="/streamprocessing1.html">What is Stream Processing?</a></li>
+ -->
+
+ <!-- What is Flink? -->
+ <li><a href="/flink-architecture.html">What is Apache Flink?</a></li>
+
+
+
+ <!-- What is Stateful Functions? -->
+
+ <li><a href="/stateful-functions.html">What is Stateful Functions?</a></li>
+
+ <!-- Use cases -->
+ <li><a href="/usecases.html">Use Cases</a></li>
+
+ <!-- Powered by -->
+ <li><a href="/poweredby.html">Powered By</a></li>
+
+
+
+ <!-- Second menu section aims to support Flink users -->
+
+ <!-- Downloads -->
+ <li><a href="/downloads.html">Downloads</a></li>
+
+ <!-- Getting Started -->
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Getting Started<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.11/getting-started/index.html" target="_blank">With Flink <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a href="https://ci.apache.org/projects/flink/flink-statefun-docs-release-2.1/getting-started/project-setup.html" target="_blank">With Flink Stateful Functions <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a href="/training.html">Training Course</a></li>
+ </ul>
+ </li>
+
+ <!-- Documentation -->
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.11" target="_blank">Flink 1.11 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-master" target="_blank">Flink Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a href="https://ci.apache.org/projects/flink/flink-statefun-docs-release-2.1" target="_blank">Flink Stateful Functions 2.1 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a href="https://ci.apache.org/projects/flink/flink-statefun-docs-master" target="_blank">Flink Stateful Functions Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ </ul>
+ </li>
+
+ <!-- getting help -->
+ <li><a href="/gettinghelp.html">Getting Help</a></li>
+
+ <!-- Blog -->
+ <li class="active"><a href="/blog/"><b>Flink Blog</b></a></li>
+
+
+ <!-- Flink-packages -->
+ <li>
+ <a href="https://flink-packages.org" target="_blank">flink-packages.org <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+
+ <!-- Third menu section aim to support community and contributors -->
+
+ <!-- Community -->
+ <li><a href="/community.html">Community & Project Info</a></li>
+
+ <!-- Roadmap -->
+ <li><a href="/roadmap.html">Roadmap</a></li>
+
+ <!-- Contribute -->
+ <li><a href="/contributing/how-to-contribute.html">How to Contribute</a></li>
+
+
+ <!-- GitHub -->
+ <li>
+ <a href="https://github.com/apache/flink" target="_blank">Flink on GitHub <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+
+
+ <!-- Language Switcher -->
+ <li>
+
+
+ <!-- link to the Chinese home page when current is blog page -->
+ <a href="/zh">中文版</a>
+
+
+ </li>
+
+ </ul>
+
+ <ul class="nav navbar-nav navbar-bottom">
+ <hr />
+
+ <!-- Twitter -->
+ <li><a href="https://twitter.com/apacheflink" target="_blank">@ApacheFlink <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <!-- Visualizer -->
+ <li class=" hidden-md hidden-sm"><a href="/visualizer/" target="_blank">Plan Visualizer <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <hr />
+
+ <li><a href="https://apache.org" target="_blank">Apache Software Foundation <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <li>
+ <style>
+ .smalllinks:link {
+ display: inline-block !important; background: none; padding-top: 0px; padding-bottom: 0px; padding-right: 0px; min-width: 75px;
+ }
+ </style>
+
+ <a class="smalllinks" href="https://www.apache.org/licenses/" target="_blank">License</a> <small><span class="glyphicon glyphicon-new-window"></span></small>
+
+ <a class="smalllinks" href="https://www.apache.org/security/" target="_blank">Security</a> <small><span class="glyphicon glyphicon-new-window"></span></small>
+
+ <a class="smalllinks" href="https://www.apache.org/foundation/sponsorship.html" target="_blank">Donate</a> <small><span class="glyphicon glyphicon-new-window"></span></small>
+
+ <a class="smalllinks" href="https://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a> <small><span class="glyphicon glyphicon-new-window"></span></small>
+ </li>
+
+ </ul>
+ </div><!-- /.navbar-collapse -->
+ </nav>
+
+ </div>
+ <div class="col-sm-9">
+ <div class="row-fluid">
+ <div class="col-sm-12">
+ <div class="row">
+ <h1>Application Deployment in Flink: current state and the new Application Mode</h1>
+ <p><i></i></p>
+
+ <article>
+ <p>14 Jul 2020 Kostas Kloudas (<a href="https://twitter.com/kkloudas">@kkloudas</a>)</p>
+
+<p>With the rise of stream processing and real-time analytics as a critical tool for modern
+businesses, an increasing number of organizations build platforms with Apache Flink at their
+core and offer it internally as a service. Many talks with related topics from companies
+like <a href="https://www.youtube.com/watch?v=VX3S9POGAdU">Uber</a>, <a href="https://www.youtube.com/watch?v=VX3S9POGAdU">Netflix</a>
+and <a href="https://www.youtube.com/watch?v=cH9UdK0yYjc">Alibaba</a> in the latest editions of Flink Forward further
+illustrate this trend.</p>
+
+<p>These platforms aim at simplifying application submission internally by lifting all the
+operational burden from the end user. To submit Flink applications, these platforms
+usually expose only a centralized or low-parallelism endpoint (<em>e.g.</em> a Web frontend)
+for application submission that we will call the <em>Deployer</em>.</p>
+
+<p>One of the roadblocks that platform developers and maintainers often mention is that the
+Deployer can be a heavy resource consumer that is difficult to provision for. Provisioning
+for average load can lead to the Deployer service being overwhelmed with deployment
+requests (in the worst case, for all production applications in a short period of time),
+while planning based on top load leads to unnecessary costs. Building on this observation,
+Flink 1.11 introduces the <a href="https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/#application-mode">Application Mode</a>
+as a deployment option, which allows for a lightweight, more scalable application
+submission process that manages to spread more evenly the application deployment load
+across the nodes in the cluster.</p>
+
+<p>In order to understand the problem and how the Application Mode solves it, we start by
+describing briefly the current status of application execution in Flink, before
+describing the architectural changes introduced by the new deployment mode and how to
+leverage them.</p>
+
+<h1 id="application-execution-in-flink">Application Execution in Flink</h1>
+
+<p>The execution of an application in Flink mainly involves three entities: the <em>Client</em>,
+the <em>JobManager</em> and the <em>TaskManagers</em>. The Client is responsible for submitting the application to the
+cluster, the JobManager is responsible for the necessary bookkeeping during execution,
+and the TaskManagers are the ones doing the actual computation. For more details please
+refer to <a href="https://ci.apache.org/projects/flink/flink-docs-stable/concepts/flink-architecture.html">Flink’s Architecture</a>
+documentation page.</p>
+
+<h2 id="current-deployment-modes">Current Deployment Modes</h2>
+
+<p>Before the introduction of the Application Mode in version 1.11, Flink allowed users to execute an application either on a
+<em>Session</em> or a <em>Per-Job Cluster</em>. The differences between the two have to do with the cluster
+lifecycle and the resource isolation guarantees they provide.</p>
+
+<h3 id="session-mode">Session Mode</h3>
+
+<p>Session Mode assumes an already running cluster and uses the resources of that cluster to
+execute any submitted application. Applications executed in the same (session) cluster use,
+and consequently compete for, the same resources. This has the advantage that you do not
+pay the resource overhead of spinning up a full cluster for every submitted job. But, if
+one of the jobs misbehaves or brings down a TaskManager, then all jobs running on that
+TaskManager will be affected by the failure. Apart from a negative impact on the job that
+caused the failure, this implies a potential massive recovery process with all the
+restarting jobs accessing the file system concurrently and making it unavailable to other
+services. Additionally, having a single cluster running multiple jobs implies more load
+for the JobManager, which is responsible for the bookkeeping of all the jobs in the
+cluster. This mode is ideal for short jobs where startup latency is of high importance,
+<em>e.g.</em> interactive queries.</p>
+
+<h3 id="per-job-mode">Per-Job Mode</h3>
+
+<p>In Per-Job Mode, the available cluster manager framework (<em>e.g.</em> YARN or Kubernetes) is
+used to spin up a Flink cluster for each submitted job, which is available to that job
+only. When the job finishes, the cluster is shut down and any lingering resources
+(<em>e.g.</em> files) are cleaned up. This mode allows for better resource isolation, as a
+misbehaving job cannot affect any other job. In addition, it spreads the load of
+bookkeeping across multiple entities, as each application has its own JobManager.
+Given the aforementioned resource isolation concerns of the Session Mode, users often
+opt for the Per-Job Mode for long-running jobs which are willing to accept some increase
+in startup latency in favor of resilience.</p>
+
+<p>To summarize, in Session Mode, the cluster lifecycle is independent of any job running on
+the cluster and all jobs running on the cluster share its resources. The per-job mode
+chooses to pay the price of spinning up a cluster for every submitted job, in order to
+provide better resource isolation guarantees as the resources are not shared across jobs.
+In this case, the lifecycle of the cluster is bound to that of the job.</p>
+
+<h2 id="application-submission">Application Submission</h2>
+
+<p>Flink application execution consists of two stages: <em>pre-flight</em>, when the users’ <code>main()</code>
+method is called; and <em>runtime</em>, which is triggered as soon as the user code calls <code>execute()</code>.
+The <code>main()</code> method constructs the user program using one of Flink’s APIs
+(DataStream API, Table API, DataSet API). When the <code>main()</code> method calls <code>env.execute()</code>,
+the user-defined pipeline is translated into a form that Flink’s runtime can understand,
+called the <em>job graph</em>, and it is shipped to the cluster.</p>
+
+<p>Despite their differences, both session and per-job modes execute the application’s <code>main()</code>
+method, <em>i.e.</em> the <em>pre-flight</em> phase, on the client side.<sup id="fnref:1"><a href="#fn:1" class="footnote">1</a></sup></p>
+
+<p>This is usually not a problem for individual users who already have all the dependencies
+of their jobs locally, and then submit their applications through a client running on
+their machine. But in the case of submission through a remote entity like the Deployer,
+this process includes:</p>
+
+<ul>
+ <li>
+ <p>downloading the application’s dependencies locally,</p>
+ </li>
+ <li>
+ <p>executing the main()method to extract the job graph,</p>
+ </li>
+ <li>
+ <p>ship the job graph and its dependencies to the cluster for execution and,</p>
+ </li>
+ <li>
+ <p>potentially, wait for the result.</p>
+ </li>
+</ul>
+
+<p>This makes the Client a heavy resource consumer as it may need substantial network
+bandwidth to download dependencies and ship binaries to the cluster, and CPU cycles to
+execute the <code>main()</code> method. This problem is even more pronounced as more users share
+the same Client.</p>
+
+<div style="line-height:60%;">
+ <br />
+</div>
+
+<center>
+<img src="/img/blog/2020-07-14-application-mode/session-per-job.png" width="75%" alt="Session and Per-Job Mode" />
+</center>
+
+<div style="line-height:150%;">
+ <br />
+</div>
+
+<p>The figure above illustrates the two deployment modes using 3 applications depicted in
+<span style="color:red">red</span>, <span style="color:blue">blue</span> and <span style="color:green">green</span>.
+Each one has a parallelism of 3. The black rectangles represent
+different processes: TaskManagers, JobManagers and the Deployer; and we assume a single
+Deployer process in all scenarios. The colored triangles represent the load of the
+submission process, while the colored rectangles represent the load of the TaskManager
+and JobManager processes. As shown in the figure, the Deployer in both per-job and
+session mode share the same load. Their difference lies in the distribution of the
+tasks and the JobManager load. In the Session Mode, there is a single JobManager for
+all the jobs in the cluster while in the per-job mode, there is one for each job. In
+addition, tasks in Session Mode are assigned randomly to TaskManagers while in Per-Job
+Mode, each TaskManager can only have tasks of a single job.</p>
+
+<h1 id="application-mode">Application Mode</h1>
+
+<p><img style="float: right;margin-left:10px;margin-right: 15px;" src="/img/blog/2020-07-14-application-mode/application.png" width="320px" alt="Application Mode" /></p>
+
+<p>The Application Mode builds on the above observations and tries to combine the resource
+isolation of the per-job mode with a lightweight and scalable application submission
+process. To achieve this, it creates a cluster <em>per submitted application</em>, but this
+time, the <code>main()</code> method of the application is executed on the JobManager.</p>
+
+<p>Creating a cluster per application can be seen as creating a session cluster shared
+only among the jobs of a particular application and torn down when the application
+finishes. With this architecture, the Application Mode provides the same resource
+isolation and load balancing guarantees as the Per-Job Mode, but at the granularity of
+a whole application. This makes sense, as jobs belonging to the same application are
+expected to be correlated and treated as a unit.</p>
+
+<p>Executing the <code>main()</code> method on the JobManager allows saving the CPU cycles required
+for extracting the job graph, but also the bandwidth required on the client for
+downloading the dependencies locally and shipping the job graph and its dependencies
+to the cluster. Furthermore, it spreads the network load more evenly, as there is one
+JobManager per application. This is illustrated in the figure above, where we have the
+same scenario as in the session and per-job deployment mode section, but this time
+the client load has shifted to the JobManager of each application.</p>
+
+<div class="alert alert-info">
+ <p><span class="label label-info" style="display: inline-block"><span class="glyphicon glyphicon-info-sign" aria-hidden="true"></span> Note</span>
+In the Application Mode, the main() method is executed on the cluster and not on the Client, as in the other modes.
+This may have implications for your code as, for example, any paths you register in your
+environment using the registerCachedFile() must be accessible by the JobManager of
+your application.</p>
+</div>
+
+<p>Compared to the Per-Job Mode, the Application Mode allows the submission of applications
+consisting of multiple jobs. The order of job execution is not affected by the deployment
+mode but by the call used to launch the job. Using the blocking <code>execute()</code> method
+establishes an order and will lead to the execution of the “next” job being postponed
+until “this” job finishes. In contrast, the non-blocking <code>executeAsync()</code> method will
+immediately continue to submit the “next” job as soon as the current job is submitted.</p>
+
+<h2 id="reducing-network-requirements">Reducing Network Requirements</h2>
+
+<p>As described above, by executing the application’s <code>main()</code> method on the JobManager,
+the Application Mode manages to save a lot of the resources previously required during
+job submission. But there is still room for improvement.</p>
+
+<p>Focusing on YARN, which already supports all the optimizations mentioned here<sup id="fnref:2"><a href="#fn:2" class="footnote">2</a></sup>, and
+even with the Application Mode in place, the Client is still required to send the user
+jar to the JobManager. In addition, <em>for each application</em>, the Client has to ship to
+the cluster the “flink-dist” directory which contains the binaries of the framework
+itself, including the <code>flink-dist.jar</code>, <code>lib/</code> and <code>plugin/</code> directories. These two can
+account for a substantial amount of bandwidth on the client side. Furthermore, shipping
+the same flink-dist binaries on every submission is both a waste of bandwidth but also
+of storage space which can be alleviated by simply allowing applications to share the
+same binaries.</p>
+
+<p>In Flink 1.11, we introduce options that allow the user to:</p>
+
+<ol>
+ <li>
+ <p>Specify a remote path to a directory where YARN can find the Flink distribution binaries, and</p>
+ </li>
+ <li>
+ <p>Specify a remote path where YARN can find the user jar.</p>
+ </li>
+</ol>
+
+<p>For 1., we leverage YARN’s distributed cache and allow applications to share these
+binaries. So, if an application happens to find copies of Flink on the local storage
+of its TaskManager due to a previous application that was executed on the same
+TaskManager, it will not even have to download it internally.</p>
+
+<div class="alert alert-info">
+ <p><span class="label label-info" style="display: inline-block"><span class="glyphicon glyphicon-info-sign" aria-hidden="true"></span> Note</span>
+Both optimizations are available to all deployment modes on YARN, and not only the Application Mode.</p>
+</div>
+
+<h1 id="example-application-mode-on-yarn">Example: Application Mode on Yarn</h1>
+
+<p>For a full description, please refer to the official Flink documentation and more
+specifically to the page that refers to your cluster management framework, <em>e.g.</em>
+<a href="https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/yarn_setup.html#run-an-application-in-application-mode">YARN</a>
+or <a href="https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/native_kubernetes.html#flink-kubernetes-application">Kubernetes</a>.
+Here we will give some examples around YARN, where all the above features are available.</p>
+
+<p>To launch an application in Application Mode, you can use:</p>
+
+<pre><code><b>./bin/flink run-application -t yarn-application</b> ./MyApplication.jar</code></pre>
+
+<p>With this command, all configuration parameters, such as the path to a savepoint to
+be used to bootstrap the application’s state or the required JobManager/TaskManager
+memory sizes, can be specified by their configuration option, prefixed by <code>-D</code>. For
+a catalog of the available configuration options, please refer to Flink’s
+<a href="https://ci.apache.org/projects/flink/flink-docs-stable/ops/config.html">configuration page</a>.</p>
+
+<p>As an example, the command to specify the memory sizes of the JobManager and the
+TaskManager would look like:</p>
+
+<pre><code>./bin/flink run-application -t yarn-application \
+ <b>-Djobmanager.memory.process.size=2048m</b> \
+ <b>-Dtaskmanager.memory.process.size=4096m</b> \
+ ./MyApplication.jar
+</code></pre>
+
+<p>As discussed earlier, the above will make sure that your application’s <code>main()</code> method
+will be executed on the JobManager.</p>
+
+<p>To further save the bandwidth of shipping the Flink distribution to the cluster, consider
+pre-uploading the Flink distribution to a location accessible by YARN and using the
+<code>yarn.provided.lib.dirs</code> configuration option, as shown below:</p>
+
+<pre><code>./bin/flink run-application -t yarn-application \
+ -Djobmanager.memory.process.size=2048m \
+ -Dtaskmanager.memory.process.size=4096m \
+ <b>-Dyarn.provided.lib.dirs="hdfs://myhdfs/remote-flink-dist-dir"</b> \
+ ./MyApplication.jar
+</code></pre>
+
+<p>Finally, in order to further save the bandwidth required to submit your application jar,
+you can pre-upload it to HDFS, and specify the remote path that points to
+<code>./MyApplication.jar</code>, as shown below:</p>
+
+<pre><code>./bin/flink run-application -t yarn-application \
+ -Djobmanager.memory.process.size=2048m \
+ -Dtaskmanager.memory.process.size=4096m \
+ -Dyarn.provided.lib.dirs="hdfs://myhdfs/remote-flink-dist-dir" \
+ <b>hdfs://myhdfs/jars/MyApplication.jar</b>
+</code></pre>
+
+<p>This will make the job submission extra lightweight as the needed Flink jars and the
+application jar are going to be picked up from the specified remote locations rather
+than be shipped to the cluster by the Client. The only thing the Client will ship to
+the cluster is the configuration of your application which includes all the
+aforementioned paths.</p>
+
+<h1 id="conclusion">Conclusion</h1>
+
+<p>We hope that this discussion helped you understand the differences between the various
+deployment modes offered by Flink and will help you to make informed decisions about
+which one is suitable in your own setup. Feel free to play around with them and report
+any issues you may find. If you have any questions or requests, do not hesitate to post
+them in the <a href="https://wints.github.io/flink-web//community.html#mailing-lists">mailing lists</a>
+and, hopefully, see you (virtually) at one of our conferences or meetups soon!</p>
+<div class="footnotes">
+ <ol>
+ <li id="fn:1">
+ <p>The only exceptions are the Web Submission and the Standalone per-job implementation. <a href="#fnref:1" class="reversefootnote">↩</a></p>
+ </li>
+ <li id="fn:2">
+ <p>Support for Kubernetes will come soon. <a href="#fnref:2" class="reversefootnote">↩</a></p>
+ </li>
+ </ol>
+</div>
+
+ </article>
+ </div>
+
+ <div class="row">
+ <div id="disqus_thread"></div>
+ <script type="text/javascript">
+ /* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE * * */
+ var disqus_shortname = 'stratosphere-eu'; // required: replace example with your forum shortname
+
+ /* * * DON'T EDIT BELOW THIS LINE * * */
+ (function() {
+ var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
+ dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js';
+ (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
+ })();
+ </script>
+ </div>
+ </div>
+</div>
+ </div>
+ </div>
+
+ <hr />
+
+ <div class="row">
+ <div class="footer text-center col-sm-12">
+ <p>Copyright © 2014-2019 <a href="http://apache.org">The Apache Software Foundation</a>. All Rights Reserved.</p>
+ <p>Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.</p>
+ <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p>
+ </div>
+ </div>
+ </div><!-- /.container -->
+
+ <!-- Include all compiled plugins (below), or include individual files as needed -->
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery.matchHeight/0.7.0/jquery.matchHeight-min.js"></script>
+ <script src="/js/codetabs.js"></script>
+ <script src="/js/stickysidebar.js"></script>
+
+ <!-- Google Analytics -->
+ <script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-52545728-1', 'auto');
+ ga('send', 'pageview');
+ </script>
+ </body>
+</html>