[jira] [Work logged] (BEAM-9161) Inconsistent view of current ActiveBundle from main and bundle timer thread

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-9161?focusedWorklogId=378430&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-378430 ]

ASF GitHub Bot logged work on BEAM-9161:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 28/Jan/20 19:40
            Start Date: 28/Jan/20 19:40
    Worklog Time Spent: 10m 
      Work Description: tweise commented on issue #10641: [BEAM-9161] Ensure non-volatile access of field variables by processing threads
URL: https://github.com/apache/beam/pull/10641#issuecomment-579419325
 
 
   Ouch, this volatile business to sync Flink managed threads is super ugly. I don't see an easy way around it until Flink 1.10, when this can be undone? https://issues.apache.org/jira/browse/FLINK-12481
   
   Perhaps add this to the comment?
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 378430)
    Time Spent: 40m  (was: 0.5h)

> Inconsistent view of current ActiveBundle from main and bundle timer thread
> ---------------------------------------------------------------------------
>
>                 Key: BEAM-9161
>                 URL: https://issues.apache.org/jira/browse/BEAM-9161
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-flink
>            Reporter: Maximilian Michels
>            Assignee: Maximilian Michels
>            Priority: Major
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> In the DoFnOperator and the subclassed ExecutableStageDoFnOperator there are
> effectively two processing threads:
> (1) The main processing thread for processing elements and watermarks
> (2) A timer thread to support ending bundles after a timeout to optimize for
> latency
> Although the code was written with a different assumption in the past, the two
> threads do not interleave with each other. Only one is active at a time. This is
> ensured by Flink's "checkpointLock" which is acquired for every method called on
> the operator like processElement, processWatermark, snapshotState. It is also
> acquired when timers are fired which are set using Flink's TimeService, like it
> is the case for (2).
> We've seen issues with bundles being closed multiple times resulting in
> exceptions like "Already closed". Very rarely, we've also seen dead bundle ids
> resurrecting for which no other explanation could be found than an inconsistent
> view of the two thread.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)