You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Beam JIRA Bot (Jira)" <ji...@apache.org> on 2020/09/15 17:08:02 UTC
[jira] [Updated] (BEAM-10493) Unbounded sources unable to recover
from checkpointMark when withMaxReadTime() is used
[ https://issues.apache.org/jira/browse/BEAM-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Beam JIRA Bot updated BEAM-10493:
---------------------------------
Labels: stale-P2 (was: )
> Unbounded sources unable to recover from checkpointMark when withMaxReadTime() is used
> --------------------------------------------------------------------------------------
>
> Key: BEAM-10493
> URL: https://issues.apache.org/jira/browse/BEAM-10493
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Affects Versions: 2.22.0
> Reporter: Mani Kolbe
> Priority: P2
> Labels: stale-P2
> Attachments: image-2020-07-15-15-15-01-705.png, image-2020-07-15-15-15-12-937.png
>
>
> Observed on v2.22.0
> When withMaxReadTime() is used, Beam creates an BoundedReadFromUnboundedSource [1]. The ReadFn<T> class in BoundedReadFromUnboundedSource which is responsible for reading records from source. You can see this class doesnt verify if there is a recoverable checkpoint exist. Instead it always creates Reader with checkpointMark set as null [2].
>
> Reference:
> [1] [https://github.com/apache/beam/blob/v2.22.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/Read.java#L205]
> [2][https://github.com/apache/beam/blob/v2.22.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BoundedReadFromUnboundedSource.java#L193]
>
> !image-2020-07-15-15-15-12-937.png|width=836,height=346!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)