You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by al...@apache.org on 2020/06/10 10:37:04 UTC
[flink] branch release-1.11 updated (e444d1f -> d939f3a)
This is an automated email from the ASF dual-hosted git repository.
aljoscha pushed a change to branch release-1.11
in repository https://gitbox.apache.org/repos/asf/flink.git.
from e444d1f [FLINK-18215][conf] Add log level to JavaBashUtils log4j config
new 3cc3310 [FLINK-16213] Move stateful-stream-processing.md introduction to form "What is State" section
new 0e080ac [FLINK-17982] Remove TODOs from stateful-stream-processing
new 973f243 [FLINK-16208] Add introduction to timely stream processing concepts documentation
new d939f3a [FLINK-17982] Remove TODOs from timely-stream-processing.md
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/concepts/stateful-stream-processing.md | 26 +++++---------------------
docs/concepts/timely-stream-processing.md | 14 +++++++++-----
2 files changed, 14 insertions(+), 26 deletions(-)
[flink] 02/04: [FLINK-17982] Remove TODOs from
stateful-stream-processing
Posted by al...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
aljoscha pushed a commit to branch release-1.11
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 0e080acea4048beb6db72f167b1ed919fe551ace
Author: Aljoscha Krettek <al...@apache.org>
AuthorDate: Tue Jun 9 12:53:55 2020 +0200
[FLINK-17982] Remove TODOs from stateful-stream-processing
For some sections, we will expand them in the future when we have actual
functionality. Some aimed at expanding sections but they are also ok as
they are now.
---
docs/concepts/stateful-stream-processing.md | 14 --------------
1 file changed, 14 deletions(-)
diff --git a/docs/concepts/stateful-stream-processing.md b/docs/concepts/stateful-stream-processing.md
index 7648791..112171a 100644
--- a/docs/concepts/stateful-stream-processing.md
+++ b/docs/concepts/stateful-stream-processing.md
@@ -61,12 +61,6 @@ provides different state backends that specify how and where state is stored.
{% top %}
-## State in Stream & Batch Processing
-
-`TODO: What is this section about? Do we even need it?`
-
-{% top %}
-
## Keyed State
Keyed state is maintained in what can be thought of as an embedded key/value
@@ -250,8 +244,6 @@ See [Restart Strategies]({% link dev/task_failure_recovery.md
### State Backends
-`TODO: expand this section`
-
The exact data structures in which the key/values indexes are stored depends on
the chosen [state backend]({% link
ops/state/state_backends.md %}). One state backend stores data in an in-memory
@@ -268,8 +260,6 @@ logic.
### Savepoints
-`TODO: expand this section`
-
All programs that use checkpointing can resume execution from a **savepoint**.
Savepoints allow both updating your programs and your Flink cluster without
losing any state.
@@ -310,10 +300,6 @@ give *exactly once* guarantees even in *at least once* mode.
{% top %}
-## End-to-end Exactly-Once Programs
-
-`TODO: add`
-
## State and Fault Tolerance in Batch Programs
Flink executes [batch programs](../dev/batch/index.html) as a special case of
[flink] 01/04: [FLINK-16213] Move stateful-stream-processing.md
introduction to form "What is State" section
Posted by al...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
aljoscha pushed a commit to branch release-1.11
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 3cc3310ae683be0b829fb67ba84d89398cdb9cf1
Author: Aljoscha Krettek <al...@apache.org>
AuthorDate: Tue Jun 9 12:52:41 2020 +0200
[FLINK-16213] Move stateful-stream-processing.md introduction to form "What is State" section
---
docs/concepts/stateful-stream-processing.md | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/docs/concepts/stateful-stream-processing.md b/docs/concepts/stateful-stream-processing.md
index b8cdd63..7648791 100644
--- a/docs/concepts/stateful-stream-processing.md
+++ b/docs/concepts/stateful-stream-processing.md
@@ -24,6 +24,11 @@ specific language governing permissions and limitations
under the License.
-->
+* This will be replaced by the TOC
+{:toc}
+
+## What is State?
+
While many operations in a dataflow simply look at one individual *event at a
time* (for example an event parser), some operations remember information
across multiple events (for example window operators). These operations are
@@ -54,13 +59,6 @@ When working with state, it might also be useful to read about [Flink's state
backends]({% link ops/state/state_backends.md %}). Flink
provides different state backends that specify how and where state is stored.
-* This will be replaced by the TOC
-{:toc}
-
-## What is State?
-
-`TODO: expand this section`
-
{% top %}
## State in Stream & Batch Processing
[flink] 04/04: [FLINK-17982] Remove TODOs from
timely-stream-processing.md
Posted by al...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
aljoscha pushed a commit to branch release-1.11
in repository https://gitbox.apache.org/repos/asf/flink.git
commit d939f3a85adc4b0a8e817c0d2a4dd4719fdf8349
Author: Aljoscha Krettek <al...@apache.org>
AuthorDate: Wed Jun 10 12:32:34 2020 +0200
[FLINK-17982] Remove TODOs from timely-stream-processing.md
For some sections, we will expand them in the future when we have actual
functionality. Some aimed at expanding sections but they are also ok as
they are now.
---
docs/concepts/timely-stream-processing.md | 6 ------
1 file changed, 6 deletions(-)
diff --git a/docs/concepts/timely-stream-processing.md b/docs/concepts/timely-stream-processing.md
index be8465f..9b96c9f 100644
--- a/docs/concepts/timely-stream-processing.md
+++ b/docs/concepts/timely-stream-processing.md
@@ -39,12 +39,6 @@ occured is important.
In the following sections we will highlight some of the topics that you should
consider when working with timely Flink Applications.
-## Latency & Completeness
-
-`TODO: add these two sections`
-
-### Latency vs. Completeness in Batch & Stream Processing
-
{% top %}
## Notions of Time: Event Time and Processing Time
[flink] 03/04: [FLINK-16208] Add introduction to timely stream
processing concepts documentation
Posted by al...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
aljoscha pushed a commit to branch release-1.11
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 973f24343911dab85456647804d3a7c15125dbf5
Author: Aljoscha Krettek <al...@apache.org>
AuthorDate: Tue Jun 9 17:14:19 2020 +0200
[FLINK-16208] Add introduction to timely stream processing concepts documentation
---
docs/concepts/timely-stream-processing.md | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/docs/concepts/timely-stream-processing.md b/docs/concepts/timely-stream-processing.md
index cb2c157..be8465f 100644
--- a/docs/concepts/timely-stream-processing.md
+++ b/docs/concepts/timely-stream-processing.md
@@ -24,11 +24,21 @@ specific language governing permissions and limitations
under the License.
-->
-`TODO: add introduction`
-
* This will be replaced by the TOC
{:toc}
+## Introduction
+
+Timely steam processing is an extension of [stateful stream processing]({% link
+concepts/stateful-stream-processing.md %}) in which time plays some role in the
+computation. Among other things, this is the case when you do time series
+analysis, when doing aggregations based on certain time periods (typically
+called windows), or when you do event processing where the time when an event
+occured is important.
+
+In the following sections we will highlight some of the topics that you should
+consider when working with timely Flink Applications.
+
## Latency & Completeness
`TODO: add these two sections`