You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by NicoK <gi...@git.apache.org> on 2018/03/29 15:54:12 UTC
[GitHub] flink pull request #5790: [FLINK-9107][docs] document timer coalescing for P...
GitHub user NicoK opened a pull request:
https://github.com/apache/flink/pull/5790
[FLINK-9107][docs] document timer coalescing for ProcessFunction
## What is the purpose of the change
In a ProcessFunction, registering timers for each event via `ctx.timerService().registerEventTimeTimer()` using timestamps like `ctx.timestamp() + timeout` will get a millisecond accuracy and may thus create one timer per millisecond which may lead to some overhead in the `TimerService`.
This problem can be mitigated by using timer coalescing if the desired accuracy of the timer can be larger than 1ms. A timer firing at full seconds only, for example, can be realised like this:
```
long coalescedTime = ((ctx.timestamp() + timeout) / 1000) * 1000;
ctx.timerService().registerEventTimeTimer(coalescedTime);
```
As a result, only a single timer may exist for every second since we do not add timers for timestamps that are already there.
Please note that this PR includes #5788 and should also be merged into 1.3 and 1.4 docs to which it applies as well.
## Brief change log
- document timer coalescing for `ProcessFunction`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NicoK/flink flink-9107
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5790.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5790
----
commit b6258697aabb8688aac19b5b228ddf8926e518f3
Author: Nico Kruber <ni...@...>
Date: 2018-03-29T13:55:47Z
[FLINK-9110][docs] fix local bundler installation
commit 540485a5620f6dcdd98e751a086076fb80997f65
Author: Nico Kruber <ni...@...>
Date: 2018-03-29T13:56:23Z
[hotfix][docs] remove duplicate bundle installation
commit f8274cf3ba06af19a4ac31e784803723e449336a
Author: Nico Kruber <ni...@...>
Date: 2018-03-29T14:20:00Z
[FLINK-9107][docs] document timer coalescing for ProcessFunction
----
---
[GitHub] flink pull request #5790: [FLINK-9107][docs] document timer coalescing for P...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/5790
---
[GitHub] flink issue #5790: [FLINK-9107][docs] document timer coalescing for ProcessF...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on the issue:
https://github.com/apache/flink/pull/5790
Actually, more advanced schemes using `current watermark + 1` (which fires with the next watermark) for the event time timer should also go into the documentation. I'll extend the PR ...
---
[GitHub] flink issue #5790: [FLINK-9107][docs] document timer coalescing for ProcessF...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on the issue:
https://github.com/apache/flink/pull/5790
thanks - fixed and extended with an example for "schedule with the next watermark" for event-time timers
---
[GitHub] flink issue #5790: [FLINK-9107][docs] document timer coalescing for ProcessF...
Posted by twalthr <gi...@git.apache.org>.
Github user twalthr commented on the issue:
https://github.com/apache/flink/pull/5790
Thank you @NicoK. I will merge this...
---
[GitHub] flink pull request #5790: [FLINK-9107][docs] document timer coalescing for P...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/5790#discussion_r178754487
--- Diff: docs/dev/stream/operators/process_function.md ---
@@ -269,4 +269,38 @@ override def onTimer(timestamp: Long, ctx: OnTimerContext, out: Collector[OUT]):
}
{% endhighlight %}
</div>
-</div>
\ No newline at end of file
+</div>
+
+## Optimisations
+
+### Timer Coalescing
+
+Every timer registered at the `TimerService` via `ctx.timerService().registerEventTimeTimer()` will
+be stored on heap and enqueued for execution. There is, however, a maximum of one timer per key and
+timestamp at a millisecond resolution and thus, in the worst case, every key may have a timer for
+each upcoming millisecond. Even if you do not do any processing for outdated timers in `onTimer`
+(as above), this may put a significant burden on the Flink runtime.
+
+Since there is only one timer per key and timestamp, however, you may coalesc timers by reducing the
--- End diff --
typo: coalesc -> coalesce
---