You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/05/12 22:59:00 UTC
[jira] [Work logged] (BEAM-12333) TimerFamilyId is not present in
TimerKey in SamzaTimerInternalsFactory and therefore doesn't get persisted
for Processing Time timers
[ https://issues.apache.org/jira/browse/BEAM-12333?focusedWorklogId=595690&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-595690 ]
ASF GitHub Bot logged work on BEAM-12333:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 12/May/21 22:58
Start Date: 12/May/21 22:58
Worklog Time Spent: 10m
Work Description: kw2542 commented on a change in pull request #14802:
URL: https://github.com/apache/beam/pull/14802#discussion_r631453131
##########
File path: runners/samza/src/main/java/org/apache/beam/runners/samza/runtime/SamzaTimerInternalsFactory.java
##########
@@ -682,16 +653,26 @@ public void encode(TimerKey<K> value, OutputStream outStream)
throws CoderException, IOException {
// encode the timestamp first
- STRING_CODER.encode(value.timerId, outStream);
- STRING_CODER.encode(value.stateNamespace.stringKey(), outStream);
+ STRING_CODER.encode(value.getTimerId(), outStream);
+ STRING_CODER.encode(value.getStateNamespace().stringKey(), outStream);
if (keyCoder != null) {
- keyCoder.encode(value.key, outStream);
+ keyCoder.encode(value.getKey(), outStream);
+ }
+
+ // encode a byte (1/0) to indicate the presence/absence of timerFamilyId
+ // We can use this approach to add additional fields in the future
+ if (!Strings.isNullOrEmpty(value.getTimerFamilyId())) {
+ BOOLEAN_CODER.encode(true, outStream);
Review comment:
could you help me understand why do we need a boolean flag? I suppose ```inStream.available() > 0``` could be sufficient to indicate there will be extra fields.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 595690)
Remaining Estimate: 0h
Time Spent: 10m
> TimerFamilyId is not present in TimerKey in SamzaTimerInternalsFactory and therefore doesn't get persisted for Processing Time timers
> -------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-12333
> URL: https://issues.apache.org/jira/browse/BEAM-12333
> Project: Beam
> Issue Type: Improvement
> Components: runner-samza
> Reporter: Ajo Thomas
> Assignee: Ajo Thomas
> Priority: P3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> `SamzaTimerInternalsFactory` stores timer state, event time and processing time timers alike, in the form of a MapState<TimerKey<K>, Long>. TimerKey, however, doesn't include the `TimerFamilyId` and is therefore not persisted. On the other hand, event time timers uses an additional time sorted set to persist the TimerData(which contains TimerFamilyId). We need to include timer family id in the TimerKey to ensure that it is persisted.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)