You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Chad Dombrova (Jira)" <ji...@apache.org> on 2019/11/01 19:47:00 UTC
[jira] [Updated] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chad Dombrova updated BEAM-6857:
--------------------------------
Description:
The Beam timers API currently requires each timer to be statically specified in the DoFn. The user must provide a separate callback method per timer. For example:
{code:java}
DoFn<String, String>()
{ @TimerId("timer1") private final TimerSpec timer1 = TimerSpecs.timer(...); @TimerId("timer2") private final TimerSpec timer2 = TimerSpecs.timer(...); ...... set timers in processElement @OnTimer("timer1") public void onTimer1() \{ .....}
@OnTimer("timer2") public void onTimer2() {....}
}
{code}
However there are many cases where the user does not know the set of timers statically when writing their code. This happens when the timer tag should be based on the data. It also happens when writing a DSL on top of Beam, where the DSL author has to create DoFns but does not know statically which timers their users will want to set (e.g. Scio).
The goal is to support dynamic timers. Something as follows;
{code:java}
DoFn<String, String>() {
@TimerId("timer") private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
@ProcessElement process(@TimerId("timer") DynamicTimer timer)
{ timer.set("tag1'", ts); timer.set("tag2", ts); }
@OnTimer("timer") public void onTimer1(@TimerTag String tag) { .....}
}
{code}
was:
The Beam timers API currently requires each timer to be statically specified in the DoFn. The user must provide a separate callback method per timer. For example:
DoFn<String, String>() {
@TimerId("timer1") private final TimerSpec timer1 = TimerSpecs.timer(...);
@TimerId("timer2") private final TimerSpec timer2 = TimerSpecs.timer(...);
...... set timers in processElement
@OnTimer("timer1") public void onTimer1() \{ .....}
@OnTimer("timer2") public void onTimer2() \{....}
}
However there are many cases where the user does not know the set of timers statically when writing their code. This happens when the timer tag should be based on the data. It also happens when writing a DSL on top of Beam, where the DSL author has to create DoFns but does not know statically which timers their users will want to set (e.g. Scio).
The goal is to support dynamic timers. Something as follows;
DoFn<String, String>() {
@TimerId("timer") private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
@ProcessElement process(@TimerId("timer") DynamicTimer timer) {
timer.set("tag1'", ts);
timer.set("tag2", ts);
}
@OnTimer("timer") public void onTimer1(@TimerTag String tag) \{ .....}
}
> Support dynamic timers
> ----------------------
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Reporter: Reuven Lax
> Assignee: Shehzaad Nakhoda
> Priority: Major
>
> The Beam timers API currently requires each timer to be statically specified in the DoFn. The user must provide a separate callback method per timer. For example:
>
> {code:java}
> DoFn<String, String>()
> { @TimerId("timer1") private final TimerSpec timer1 = TimerSpecs.timer(...); @TimerId("timer2") private final TimerSpec timer2 = TimerSpecs.timer(...); ...... set timers in processElement @OnTimer("timer1") public void onTimer1() \{ .....}
> @OnTimer("timer2") public void onTimer2() {....}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers statically when writing their code. This happens when the timer tag should be based on the data. It also happens when writing a DSL on top of Beam, where the DSL author has to create DoFns but does not know statically which timers their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn<String, String>() {
> @TimerId("timer") private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> { timer.set("tag1'", ts); timer.set("tag2", ts); }
> @OnTimer("timer") public void onTimer1(@TimerTag String tag) { .....}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)