[GitHub] [beam] kennknowles opened a new issue, #18426: Watermark can become unavailable for executors while it's updated with new values

[GitBox <gi...@apache.org>]

kennknowles opened a new issue, #18426:
URL: https://github.com/apache/beam/issues/18426

   The watermark is updated by the driver like so:
   
   ```
   
   blockManager.removeBlock(WATERMARKS_BLOCK_ID, true);
   blockManager.putSingle(WATERMARKS_BLOCK_ID, newValues,
   StorageLevel.MEMORY_ONLY(), 
   true);
   
   ```
   
   
   However, these operations are neither synchronous nor atomic, so if an executor requests the watermark values before they are actually put but after they have been removed, it may get a `null` as a response, which will make it default to negative infinity as the watermark. This can result in an erroneous results.
   
   To overcome this in tests, a workaround which assumes a single JVM setting is used. In such a setting the watermark values are stored in a static member, accessible by both the driver and the executors, bypassing the `BlockManager#putSingle(...)` `BlockManager#removeBlock(...)` APIs.
   
   Imported from Jira [BEAM-2789](https://issues.apache.org/jira/browse/BEAM-2789). Original Jira may contain additional context.
   Reported by: staslev.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org