You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by alpinegizmo <gi...@git.apache.org> on 2018/05/02 11:57:02 UTC
[GitHub] flink pull request #5949: [FLINK-9288][docs] clarify the event time / waterm...
GitHub user alpinegizmo opened a pull request:
https://github.com/apache/flink/pull/5949
[FLINK-9288][docs] clarify the event time / watermark docs
This PR only affects the documentation (for event time and watermarks). I wanted to make a couple of things clearer, and to provide a couple of additional internal links.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/alpinegizmo/flink event-time-watermarks-docs
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5949.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5949
----
commit a5c66a56845ada7b0a20471a4842175c5a6566d6
Author: David Anderson <da...@...>
Date: 2018-05-02T11:50:48Z
[FLINK-9288][docs] clarify the event time / watermark docs
----
---
[GitHub] flink issue #5949: [FLINK-9288][docs] clarify the event time / watermark doc...
Posted by alpinegizmo <gi...@git.apache.org>.
Github user alpinegizmo commented on the issue:
https://github.com/apache/flink/pull/5949
@bowenli86 Thanks for the feedback. I've reworked that event time section. Hopefully it's now more complete and accurate without being too complex.
---
[GitHub] flink pull request #5949: [FLINK-9288][docs] clarify the event time / waterm...
Posted by bowenli86 <gi...@git.apache.org>.
Github user bowenli86 commented on a diff in the pull request:
https://github.com/apache/flink/pull/5949#discussion_r185588020
--- Diff: docs/dev/event_time.md ---
@@ -35,30 +35,32 @@ Flink supports different notions of *time* in streaming programs.
respective operation.
When a streaming program runs on processing time, all time-based operations (like time windows) will
- use the system clock of the machines that run the respective operator. For example, an hourly
+ use the system clock of the machines that run the respective operator. An hourly
processing time window will include all records that arrived at a specific operator between the
- times when the system clock indicated the full hour.
+ times when the system clock indicated the full hour. For example, if an application
+ begins running at 9:15am, the first hourly processing time window will include events
+ processed between 9:15am and 10:00am, the next window will include events processed between 10:00am and 11:00am, and so on.
Processing time is the simplest notion of time and requires no coordination between streams and machines.
It provides the best performance and the lowest latency. However, in distributed and asynchronous
environments processing time does not provide determinism, because it is susceptible to the speed at which
- records arrive in the system (for example from the message queue), and to the speed at which the
- records flow between operators inside the system.
+ records arrive in the system (for example from the message queue), to the speed at which the
+ records flow between operators inside the system, and to outages (scheduled, or otherwise).
- **Event time:** Event time is the time that each individual event occurred on its producing device.
- This time is typically embedded within the records before they enter Flink and that *event timestamp*
- can be extracted from the record. An hourly event time window will contain all records that carry an
- event timestamp that falls into that hour, regardless of when the records arrive, and in what order
- they arrive.
+ This time is typically embedded within the records before they enter Flink, and that *event timestamp*
+ can be extracted from each record. An hourly event time window will contain all records that carry an
--- End diff --
better mention allowed lateness here. “...will contain all records, ..., regardless of when the records arrive” sounds too absolute, the guarantee can only be achieved with lateness requirements
---
[GitHub] flink pull request #5949: [FLINK-9288][docs] clarify the event time / waterm...
Posted by alpinegizmo <gi...@git.apache.org>.
GitHub user alpinegizmo reopened a pull request:
https://github.com/apache/flink/pull/5949
[FLINK-9288][docs] clarify the event time / watermark docs
This PR only affects the documentation (for event time and watermarks). I wanted to make a couple of things clearer, and to provide a couple of additional internal links.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/alpinegizmo/flink event-time-watermarks-docs
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5949.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5949
----
commit a5c66a56845ada7b0a20471a4842175c5a6566d6
Author: David Anderson <da...@...>
Date: 2018-05-02T11:50:48Z
[FLINK-9288][docs] clarify the event time / watermark docs
commit a100cab6fec6ab3affa4ecc13c46e0081bd19b62
Author: David Anderson <da...@...>
Date: 2018-05-03T05:12:12Z
Reworked the section on event time to be less absolutist.
----
---
[GitHub] flink issue #5949: [FLINK-9288][docs] clarify the event time / watermark doc...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/5949
Looks good, thanks, merging this...
---
[GitHub] flink pull request #5949: [FLINK-9288][docs] clarify the event time / waterm...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/5949
---
[GitHub] flink pull request #5949: [FLINK-9288][docs] clarify the event time / waterm...
Posted by alpinegizmo <gi...@git.apache.org>.
Github user alpinegizmo closed the pull request at:
https://github.com/apache/flink/pull/5949
---